No localization with Livox Lidar about direct_lidar_inertial_odometry HOT 46 CLOSED

@@yES! DLIO is now working with deskewing！just modify livox_topic to "/livox/lidar"

from direct_lidar_inertial_odometry.

@Joosoo1 what are the extrinsics of the dataset you provided?
Please use this dataset, I tested it and it works great!：https://1drv.ms/u/s!AjkIEg1kWUNeiArwedvaRRYd5hWB?e=BaII3v
The extrinsics are as follows：

from direct_lidar_inertial_odometry.

It will undistort the point cloud when set to true.

Thanks

from direct_lidar_inertial_odometry.

Yeah I haven't tested Livox support out since I don't have any data. Think you could upload a bag for me to play around with? Thanks!

from direct_lidar_inertial_odometry.

This is the dataset of Livox-mid360: https://1drv.ms/u/s!AjkIEg1kWUNeiAIZHIDBTzXmtHqf?e=cWek2u
And the extrinsic between the Lidar frame and IMU frame is

from direct_lidar_inertial_odometry.

Yes, it seems that we are experiencing the same problem, may I ask if you are using it as a navigation for robots? Drones or unmanned vehicles?Maybe we can communicate that i have recently encountered some difficulties.

from direct_lidar_inertial_odometry.

Yeah I 'm going to use it as a localization module for drones, and maybe we can use the g-mail to communicate: [email protected]

Yes, it seems that we are experiencing the same problem, may I ask if you are using it as a navigation for robots? Drones or unmanned vehicles?Maybe we can communicate that i have recently encountered some difficulties.↳

from direct_lidar_inertial_odometry.

ok, after my test, livox-mid360 can run DLIO successfully, just change the point cloud publishing form "xfer_format" to "2" in livox_ros_driver2, then it can run DLIO successfully. DLIO can be run successfully, as follows:

from direct_lidar_inertial_odometry.

But the results do not seem to be very good, odom data z-axis jitter will be relatively large, may be due to the reason of not tuning parameters, but also may be due to the built-in imu noise is too large, the follow-up we can exchange about how to make it better and more stable, in addition, I am also intended to apply it to the top of the drone, used as navigation and positioning.

from direct_lidar_inertial_odometry.

Thanks to your reply, the dimension of the acceleration data from the build-in imu in Livox MID-360 is g(gravity) instead of m/s^2, maybe you should multiply the data with a constant value like 9.8?

from direct_lidar_inertial_odometry.

Thanks to your reply, the dimension of the acceleration data from the build-in imu in Livox MID-360 is g(gravity) instead of m/s^2, maybe you should multiply the data with a constant value like 9.8?

ok,i will try!If you have better results, please let me know too!

from direct_lidar_inertial_odometry.

I have tried to set xfer_format t o 2, and the result seems great!

from direct_lidar_inertial_odometry.

wow! Your results look great, the position estimation is accurate and the built point cloud map has no drift, but my results are as bad as ever, still with a large offset on the z-axis, can you share your configuration parameters file? Also, about the z-axis, my livox has a z-axis accel bias of -5, while yours is 0. I think there is something wrong here.

from direct_lidar_inertial_odometry.

Hi @WeilongXia and @Joosoo1 -- thanks for the great discussion you two have started here. Looks like the Livox branch is working! It just needs to be published in the standard format.

Yeah something is definitely wrong if the IMU biases are that large. Have you checked your extrinsics? That's typically an issue, especially since the convention for IMU and LiDAR are different.

Edit: it's working, but deskewing is not (note the deskewed points: 0 in the terminal screenshots above). Looks like a per-point timestamp issue with Livox sensors.

from direct_lidar_inertial_odometry.

@Joosoo1 Yes, change the units of the accelerometer to m/s^2 as per the advice above.

Could someone record me a dataset in the new xfer_format? Thanks.

from direct_lidar_inertial_odometry.

Well, it's true that because of the acceleration units, it has to be m/s^2 for it to work well.
I provide the dataset：https://1drv.ms/u/s!AiyqJY-cvK5varx4IwdfshuTfeE?e=TtbAdI
pointcloud_topic:/livox/lidar
imu_topic:/livox/imu (the acceleration unit is g, so you need to modify the code in DLIO)
The parameters are:
! livox-mid360 (built-in imu) extranet

from direct_lidar_inertial_odometry.

Can you translate in English Please!. I am trying to run Livox but its not working

from direct_lidar_inertial_odometry.

Can you translate in English Please!. I am trying to run Livox but its not working
You can take a look at this: #5 (comment)

It is necessary to additionally convert the acceleration units of the /livox/imu topic output to m/s^2,then the DLIO will work well, of course all this needs to be under the source code of the livox branch.

from direct_lidar_inertial_odometry.

@kennyjchen I found that when livox outputs the standard point cloud, its feild.name is not "offset_time", so according to the code "deskew_= false", deskewing is not wrok!

from direct_lidar_inertial_odometry.

Hmm, yeah unfortunately it looks like in the livox_ros_driver2 package, only x y z and intensity fields are copied over when publishing using the xfer_format: 2 value.

https://github.com/Livox-SDK/livox_ros_driver2/blob/master/src/lddc.cpp#L437

It looks like with their custom message, they have per-point timestamps . However, it looks like both timestamp and offset_time are used, which is a bit confusing since their CustomPoint.msg contains offset_time, but the data above (the first link) has a timestamp field.

If only timestamps are available using their custom point format, we'll need to build support for that (unless they add timestamps to the PCL format). I'll keep digging.

from direct_lidar_inertial_odometry.

Ok -- I think I might have figured this out. For future reference, this is how different sensors timestamp their points:

• Ouster
  • field t of type std::uint32_t, which is the time relative to the beginning of the scan in nanoseconds
• Velodyne
  • field time of type float, which is the time relative to the beginning of the scan in seconds
• Livox
  • xfer_format: 0: field timestamp of type double, which is the absolute time of the point since epoch in nanoseconds
  • xfer_format: 1: field offset_time of type std::uint32_t, which is the time relative to the beginning of the scan in nanoseconds
• Hesai
  • field timestamp of type double, which is the absolute time of the point since epoch in seconds

I pushed a few changes to the feature/livox-support branch. Please test this out using the original xfer_format value so that there are timestamp fields.

@WeilongXia -- I played around with your data, and it looks like the LiDAR and IMU measurements are not time synchronized. Is there an option in the Livox driver to synchronize them somehow?

EDIT 6/28: updated the list above to be more accurate and include both xfer_format: 0 and xfer_format: 1. As of the time of this writing, seems like there is a bug in timestamping with xfer_format: 0.

from direct_lidar_inertial_odometry.

Ok, I found a timesync choice in the config file like this

and I'll try to record a new rosbag.

from direct_lidar_inertial_odometry.

@WeilongXia could you rename the issue to something more meaningful, if you don't mind, for future reference? Something like "No localization with Livox Lidar"

from direct_lidar_inertial_odometry.

I tested the feature/livox-support branch with the original xfer_format(value 1) , but there is no localization output. @Joosoo1 Have you tested the new branch with xfer_format value is 1?
In livox_ros_driver2, when I run the msg_MID360.launch file, the lidar topic type is livox_ros_driver2/CustomMsg rather than sensor_msgs::PointCloud2, so I modify some code in livox_ros_driver2 like this:

else if (kLivoxCustomMsg == transfer_format_)
        {
            // PublishCustomPointcloud(p_queue, index);
            PublishPointcloud2(p_queue, index);
        }

else if (kLivoxCustomMsg == transfer_format_)
        {
            // **pub = cur_node_->GetNode().advertise<livox_ros_driver2::CustomMsg>(name_str, queue_size);
            // DRIVER_INFO(*cur_node_, "%s publish use livox custom format, set ROS publisher queue size %d", name_str,
            //             queue_size);
            **pub = cur_node_->GetNode().advertise<sensor_msgs::PointCloud2>(name_str, queue_size);
            DRIVER_INFO(*cur_node_, "%s publish use PointCloud2 format, set ROS publisher queue size %d", name_str,
                        queue_size);
        }

else if (kLivoxCustomMsg == msg_type)
    {
        // DRIVER_INFO(*cur_node_, "%s publish use livox custom format", topic_name.c_str());
        // return cur_node_->create_publisher<CustomMsg>(topic_name, queue_size);
        DRIVER_INFO(*cur_node_, "%s publish use PointCloud2 format", topic_name.c_str());
        return cur_node_->create_publisher<PointCloud2>(topic_name, queue_size);
    }

from direct_lidar_inertial_odometry.

ld multiply the data with a constant value like 9.8?

@Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other

from direct_lidar_inertial_odometry.

Yes, I tested the new branch and the DLIO only has odom output when xfer_format=2, but the result is completely wrong, so I have been using the original DLIO now, the original DLIO works well, I tested it on the drone and the positioning is very good. But deskwing is not working and will have some effect on DLIO，So I look forward to the author's help in solving this problem.

from direct_lidar_inertial_odometry.

ld multiply the data with a constant value like 9.8?

@Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other
xfer_format = 2, old DLIO could working!

from direct_lidar_inertial_odometry.

ld multiply the data with a constant value like 9.8?

@Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other
xfer_format = 2, old DLIO could working!

Also, when you use imu with acceleration units of m/s^2, you should change the https://github.com/vectr-ucla/direct_lidar_inertial_odometry/tree/feature/livox-support/src/ under the livox branch Lines 1373 to 1375 of the code in dlio/odom.cc should be changed to Eigen::Vector3f lin_accel(imu_raw->linear_acceleration.x * gravity_, imu_raw->linear_acceleration.y * gravity_, imu_raw->linear_acceleration.z * gravity_); recompile！

from direct_lidar_inertial_odometry.

from direct_lidar_inertial_odometry.

Thanks for your reply and kind advice. I have another problem when compiling, the first is the version of cmake. As my ros version is melodic,and the cmake version is 3.10.2, so I modify #VERSION 3.12.4 cmake_minimum_required(VERSION 3.10.2) project(direct_lidar_inertial_odometry) Could this modification do influence the output of the code?  it have no add_compile_definitions.Then I comment out the section:# Not all machines have <cpuid.h> available #set(CMAKE_TRY_COMPILE_TARGET_TYPE STATIC_LIBRARY) #file(WRITE ${CMAKE_BINARY_DIR}/test_cpuid.cpp "#include <cpuid.h>") #try_compile(HAS_CPUID ${CMAKE_BINARY_DIR} ${CMAKE_BINARY_DIR}/test_cpuid.cpp) #file(REMOVE ${CMAKE_BINARY_DIR}/test_cpuid.cpp) #if(HAS_CPUID) #  add_compile_definitions(HAS_CPUID) #endif() after done above, I compiled successfully.
…
------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2023年6月26日(星期一) 中午11:54 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [vectr-ucla/direct_lidar_inertial_odometry] No localization with Livox Lidar (Issue #5) ld multiply the data with a constant value like 9.8? @Joosoo1 I used xfer_format = 0,and there was no point cloud. My wechat:yijingxs, you can add me and it is convenient to communicate with each other xfer_format = 2, old DLIO could working! Also, when you use imu with acceleration units of m/s^2, you should change the https://github.com/vectr-ucla/direct_lidar_inertial_odometry/tree/feature/livox-support/src/ under the livox branch Lines 1373 to 1375 of the code in dlio/odom.cc should be changed to Eigen::Vector3f lin_accel(imu_raw->linear_acceleration.x * gravity_, imu_raw->linear_acceleration.y * gravity_, imu_raw->linear_acceleration.z * gravity_); recompile！ — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

I don't know much about this, but if it compiles successfully, you should be able to run the program successfully, so you can test it as soon as possible!

from direct_lidar_inertial_odometry.

@yijing2050 Unfortunately DLIO only supports ROS Noetic, but it sounds like you got it working. Removing those lines won't have any affect on the output; it's just to build support for ARM architectures.

@WeilongXia @Joosoo1 The xfer_format should be 0 so that it's a message of type PointCloud2 but contains the timestamps. I will look into this again later this week. Sorry to ask again -- could someone record me a new bag that ideally time synchronizes the Mid-360's LiDAR and IMU (if not already), using xfer_format: 0? Thank you. In the meanwhile, you can turn deskewing off, but note that (especially at 10Hz) the point clouds will be quite distorted.

from direct_lidar_inertial_odometry.

@kennyjchen
livox_repub.zip
Uploading livox_repub.zip…

from direct_lidar_inertial_odometry.

Actually, I have compiled successfully although, but i haven't got the odom.. The configuration is as the below.

from direct_lidar_inertial_odometry.

Ok -- after digging through the Livox driver and also looking at this dataset, it seems like there is a bug for per-point timestamps when using xfer_format: 0(i.e., Livox pointcloud2(PointXYZRTLT) pointcloud format). The point cloud and IMU timestamps look synchronized, but the point timestamps are off and do not fall within the sweep window for some reason. This might be just the provided dataset, but either way they don't look lined-up.

I looked at other Mid-360 packages (e.g., this), and it seems like most people use xfer_format: 1 (Livox's custom pointcloud format) so that the offset_time field is available and contains the time since the beginning of the sweep. This requires a subscription to Livox's custom pointcloud format though.

So new plan: please try the updated branch but with xfer_format: 1. Your topic of type livox_ros_driver::CustomMsg should be called /robot/livox, which will call this new callback. This is all assuming that offset_time is in nanoseconds relative to the beginning of the sweep (which I think is the case).

(Please be careful with the units of gravity. I removed the previous multiplication with gravity.)

Note that I have not tested this at all, because I do not have a Livox sensor or any bags with xfer_format: 1. If someone could record me one that would be great so I can test it on my side. Otherwise, let me know if it works or not!

from direct_lidar_inertial_odometry.

Sorry for the late reply, and thank you @kennyjchen author for your efforts to solve this problem. Here is the dataset I recorded: accel in units is g；and xfer_format: 1
https://onedrive.live.com/?cid=6FAEBC9C8F25AA2C&id=6FAEBC9C8F25AA2C%21120&parId=root&o=OneUp

from direct_lidar_inertial_odometry.

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.

from direct_lidar_inertial_odometry.

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.

Doesn't seem to work：

from direct_lidar_inertial_odometry.

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.↳

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.↳

Doesn't seem to work： ↳

You should modify livox_topic to "/livox/lidar", and pointcloud_topic to another one.
Hi author @kennyjchen , with deskew set to "false", it can work with Livox-MID360 now as below!

And the offset_time units in livox_ros_driver2 is microseconds.

from direct_lidar_inertial_odometry.

Hi author, when i set deskew to true, the program is not working properly, this is the dataset i used: https://1drv.ms/u/s!AjkIEg1kWUNeiArwedvaRRYd5hWB?e=BaII3v
Could you please help to figure this out?

from direct_lidar_inertial_odometry.

@WeilongXia -- Your dataset works for me with deskewing.

I'm using the latest feature/livox-support branch without any modifications and the following command:

rosbag play 2023-06-28-10-00-30.bag /livox/imu:=/robot/imu /livox/lidar:=/robot/livox

What are the errors that you're seeing?

from direct_lidar_inertial_odometry.

Good to hear!

from direct_lidar_inertial_odometry.

DLIO is now working with deskewing in Ubuntu 18.04, but it seems not stable and the path is not smooth as fast_lio2. And the map will fly in some time later.

from direct_lidar_inertial_odometry.

That max comp time looks pretty awful, which can result in these sorts of behaviors if too many frames are skipped. Are extrinsics correct?

from direct_lidar_inertial_odometry.

@Joosoo1 what are the extrinsics of the dataset you provided?

from direct_lidar_inertial_odometry.

May I ask you guys what is the meaning of dlio/pointcloud/dekew in dlio.yaml? Does it mean that this program will do the deskewing for pointcloud, or this program will consider the input points has been deskewed so it will no do the deskewing? What will happen when dlio/pointcloud/dekew is set as false? Thank you!

Ok, I think I got it working. Credit to this Tweet for supplying the data I played around with.↳

Please use the newer livox_ros_driver2 package with xfer_format: 1. The latest feature/livox-support branch seems to be working with deskewing. Let me know.↳

Doesn't seem to work： ↳

You should modify livox_topic to "/livox/lidar", and pointcloud_topic to another one. Hi author @kennyjchen , with deskew set to "false", it can work with Livox-MID360 now as below! And the offset_time units in livox_ros_driver2 is microseconds.

from direct_lidar_inertial_odometry.

It will undistort the point cloud when set to true.

from direct_lidar_inertial_odometry.