appinho / sarosperceptionkitti Goto Github PK

ROS package for the Perception (Sensor Processing, Detection, Tracking and Evaluation) of the KITTI Vision Benchmark Suite

License: MIT License

CMake 14.27% Python 44.98% C++ 34.52% MATLAB 6.24%

ros ros-kinetic ros-packages rviz rosbag semantic-segmentation sensor-fusion multi-object-tracking cpp python

sarosperceptionkitti's Introduction

SARosPerceptionKitti

ROS package for the Perception (Sensor Processing, Detection, Tracking and Evaluation) of the KITTI Vision Benchmark

Demo

Setup

Sticking to this folder structure is highly recommended:

    ~                                        # Home directory
    ├── catkin_ws                            # Catkin workspace
    │   ├── src                              # Source folder
    │       └── SARosPerceptionKitti         # Repo
    ├── kitti_data                           # Dataset
    │   ├── 0012                             # Demo scenario 0012
    │   │   └── synchronized_data.bag        # Synchronized ROSbag file

Install ROS and create a catkin workspace in your home directory:

mkdir -p ~/catkin_ws/src

Clone this repository into the catkin workspace's source folder (src) and build it:

cd ~/catkin_ws/src
git clone https://github.com/appinho/SARosPerceptionKitti.git
cd ~/catkin_ws
catkin_make
source devel/setup.bash

Download a preprocessed scenario and unzip it into a separate kitti_data directory, also stored under your home directory:

mkdir ~/kitti_data && cd ~/kitti_data/
mv ~/Downloads/0012.zip .
unzip 0012.zip
rm 0012.zip

Usage

Launch one of the following ROS nodes to perform and visualize the pipeline (Sensor Processing -> Object Detection -> Object Tracking) step-by-step:

source devel/setup.bash
roslaunch sensor_processing sensor_processing.launch home_dir:=/home/YOUR_USERNAME
roslaunch detection detection.launch home_dir:=/home/YOUR_USERNAME
roslaunch tracking tracking.launch home_dir:=/home/YOUR_USERNAME

Default parameters:
- scenario:=0012
- speed:=0.2
- delay:=3

Without assigning any of the abovementioned parameters the demo scenario 0012 is replayed at 20% of its speed with a 3 second delay so RViz has enough time to boot up.

Write the results to file and evaluate them:

roslaunch evaluation evaluation.launch home_dir:=/home/YOUR_USERNAME
cd ~/catkin_ws/src/SARosPerceptionKitti/benchmark/python
python evaluate_tracking.py

Results for demo scenario 0012

Class	MOTA	MOTP	MOTAL	MODA	MODP
Car	0.881119	0.633595	0.881119	0.881119	0.642273
Pedestrian	0.546875	0.677919	0.546875	0.546875	0.836921

Contact

If you have any questions, things you would love to add or ideas how to actualize the points in the Area of Improvements, send me an email at [email protected] ! More than interested to collaborate and hear any kind of feedback.

sarosperceptionkitti's People

Contributors

Stargazers

Watchers

Forkers

spacefan kejingjing firecloudy changandao collector-m idontlikelongname xiaohedu yangdaiyu123 akash-sachan hiwad-aziz csmanu liaojiacai andrewraharjo chenbaifan luis2r jediofgever autolidarperception sxontheway tyunist hgp2huge karolmajek cristianku songanz zhi3 ankushbraj dingjiangang sakura1221 hitleon sevenchao ywyue hzm8341 mzhian htang2012 wutaoo smahajan07 fenix0817 aoyuqc joserfjuniorllms hywel1994 zhouzhuol yiboliu31 xiaoliangabc algnersyjw kevin2599 liangshaoteng lhzlhz arslan-z zaiwangli scud3r1a ginson shanpenghui njauwangkai vega4 weiweizhang6338 cassiel829 gxq0527 imdsafi09 nana0127 rgbitx songruiqi seo-dev cule szymonpa nattapas lijiunderstand atreyabhat hdm8809 watchhub canihaveaname atinge haitian-zhang sysy98 yznmur hoangduyloc marufa2018 inguchoi cristianrodrigoguarachiibanez pxu2018 pandalehre

sarosperceptionkitti's Issues

Missing documentation

Thanks for the sharing the awesome work, I'd like to replicate this and maybe use this in conjunction with voxelnet. It'll be great if the current README could be updated so that I work with this.

track algorithm

Hi， I want to ask why tracks_[i].x(2) is less than 0 which should be the velocity.

Failed to launch evaluation evaluation.launch

I have run roslaunch evaluation evaluation.launch scenario:=0001 and I got a warning like this

[ INFO] [1553662220.801529193]: Read scenario : 0001
[ WARN] [1553662223.935368898]: Error opening file
[ INFO] [1553662223.935418984]: Publishing Evaluation [0]
[ WARN] [1553662224.441130598]: Error opening file
[ INFO] [1553662224.441191681]: Publishing Evaluation [1]
[ WARN] [1553662224.958780696]: Error opening file

Then, I check the evaluation.cpp, I found there is no kitti_results folder. Would you please upload it?
Thank you very much!

// Delete content in file if there is one
	filename_ = 
		"~/kitti_results/"
		+ scenario_name + ".txt";
	tracking_results_.open(filename_.c_str(),
		std::ofstream::out | std::ofstream::trunc);

where is the 0060.zip

Where can I download the 0060.zip? I used 0006.zip but only see radar scans in Rviz without images.

Did anyone try running ukf without world coordinate?

Hello! I've tried to modify your code to suit my project —— tracking vehicles on highway.
It works but there are also some issues ： I cannot get the conversion from vehicle coordinate system to world coordinate system so that I only run code on vehicle coordinate system， and the bounding box, the velocity and yaw of vehicle also seem strange.
Did you try running ukf without world coordinate?And any advise?
Thanks a lot!

[detection.launch] is neither a launch file in package [detection] nor is [detection] a launch file name [tracking.launch] is neither a launch file in package [tracking] nor is [tracking] a launch file nam

Hi,

When i executed
roslaunch detection detection.launch home_dir:=/home/sana
roslaunch tracking tracking.launch home_dir:=/home/sana

i got the error below: Kindly help
[detection.launch] is neither a launch file in package [detection] nor is [detection] a launch file name
The traceback for the exception was written to the log file

[tracking.launch] is neither a launch file in package [tracking] nor is [tracking] a launch file name
The traceback for the exception was written to the log file

Empty semantic point cloud!

downloaded the data and ran everything exactly the smae and I don't get objects.
I only get "Empty semantic point cloud!"
And I also noticed that the folder
"/kitti_data/0012/segmented_semantic_images/" is one of the folders that is needed.

DO I need to download and make the Kitti Data manually?

Question regarding launch of project

Hi
I followed the exact instructions for launching demo and downloaded 0012.zip file and extracted it in ~/kitti_data/ as stated in step 3 demo instruction. when this

roslaunch sensor_processing sensor_processing.launch home_dir:=/home/user

command is entered I get this output:

I searched in previous issues and found out that step 3 is actually wrong and the folder structure of kitti_data/0012.bag should be something like ~/kitti_data/0012/synchronized_data.bag and also it should have supplementary folder for images which I don't know how should I grab it.

For each scenario we should have a folder named scenario number and inside it a synchronized_data.bag file and a folder with images.

with above changes I could get the following output by entering first roslaunch sensor_processing sensor_processing.launch home_dir:=/home/user command:

and this result for roslaunch tracking tracking.launch home_dir:=/home/user:

Is this output correct?

I am wondering what should I do in order to get a better (in terms of visual) output? What am I doing wrong? I want to see a similar output that you have in README file.

Unfortunately the google drive link only have files named 0000.bag, 0001.bag and ...

I want to add that I am pretty new to this stuff so I hope you don't mind my basic questions.

Regards
Hadi

Project not working with my own converted kitti dataset

I downloaded another dataset with the help of following commands:

$ wget https://s3.eu-central-1.amazonaws.com/avg-kitti/raw_data/2011_09_26_drive_0002/2011_09_26_drive_0002_sync.zip
$ wget https://s3.eu-central-1.amazonaws.com/avg-kitti/raw_data/2011_09_26_calib.zip
$ unzip 2011_09_26_drive_0002_sync.zip
$ unzip 2011_09_26_calib.zip
$ kitti2bag -t 2011_09_26 -r 0002 raw_synced .

and this is the output of rosbag info:

user@user:~/kitti_data/0002$ rosbag info synchronized_data.bag 
path:        synchronized_data.bag
version:     2.0
duration:    7.8s
start:       Sep 26 2011 13:02:44.33 (1317029564.33)
end:         Sep 26 2011 13:02:52.16 (1317029572.16)
size:        417.3 MB
messages:    1078
compression: none [308/308 chunks]
types:       geometry_msgs/TwistStamped [98d34b0043a2093cf9d9345ab6eef12e]
             sensor_msgs/CameraInfo     [c9a58c1b0b154e0e6da7578cb991d214]
             sensor_msgs/Image          [060021388200f6f0f447d0fcd9c64743]
             sensor_msgs/Imu            [6a62c6daae103f4ff57a132d6f95cec2]
             sensor_msgs/NavSatFix      [2d3a8cd499b9b4a0249fb98fd05cfa48]
             sensor_msgs/PointCloud2    [1158d486dd51d683ce2f1be655c3c181]
             tf2_msgs/TFMessage         [94810edda583a504dfda3829e70d7eec]
topics:      /kitti/camera_color_left/camera_info    77 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_color_left/image_raw      77 msgs    : sensor_msgs/Image         
             /kitti/camera_color_right/camera_info   77 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_color_right/image_raw     77 msgs    : sensor_msgs/Image         
             /kitti/camera_gray_left/camera_info     77 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_gray_left/image_raw       77 msgs    : sensor_msgs/Image         
             /kitti/camera_gray_right/camera_info    77 msgs    : sensor_msgs/CameraInfo    
             /kitti/camera_gray_right/image_raw      77 msgs    : sensor_msgs/Image         
             /kitti/oxts/gps/fix                     77 msgs    : sensor_msgs/NavSatFix     
             /kitti/oxts/gps/vel                     77 msgs    : geometry_msgs/TwistStamped
             /kitti/oxts/imu                         77 msgs    : sensor_msgs/Imu           
             /kitti/velo/pointcloud                  77 msgs    : sensor_msgs/PointCloud2   
             /tf                                     77 msgs    : tf2_msgs/TFMessage        
             /tf_static                              77 msgs    : tf2_msgs/TFMessage

and when I ran this command at command prompt:

user@user:~/catkin_ws$ roslaunch sensor_processing sensor_processing.launch home_dir:=/home/user

I get this result in terminal and rviz shows nothing!

user@user:~/catkin_ws$ roslaunch sensor_processing sensor_processing.launch home_dir:=/home/user
... logging to /home/user/.ros/log/42b8fb0c-864e-11ea-92dd-240a641f2dc5/roslaunch-user-8760.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://user:35757/

SUMMARY
========

PARAMETERS
 * /rosdistro: melodic
 * /rosversion: 1.14.5
 * /sensor_setup_node/grid/cell/height: 0.15
 * /sensor_setup_node/grid/cell/size: 0.25
 * /sensor_setup_node/grid/range/max: 83.0
 * /sensor_setup_node/grid/range/min: 3.0
 * /sensor_setup_node/grid/segments: 240
 * /sensor_setup_node/home_dir: /home/user
 * /sensor_setup_node/lidar/height: -1.73
 * /sensor_setup_node/lidar/z_min: -2.4
 * /sensor_setup_node/ransac/iterations: 50
 * /sensor_setup_node/ransac/tolerance: 0.2
 * /sensor_setup_node/scenario: 2
 * /sensor_setup_node/semantic/edge_detection/kernel: 3
 * /sensor_setup_node/semantic/edge_detection/max: 120
 * /sensor_setup_node/semantic/edge_detection/min: 5
 * /sensor_setup_node/semantic/edge_detection/perform: False

NODES
  /
    my_rviz (rviz/rviz)
    player (rosbag/play)
    sensor_setup_node (sensor_processing/sensor_setup)

auto-starting new master
process[master]: started with pid [8771]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 42b8fb0c-864e-11ea-92dd-240a641f2dc5
process[rosout-1]: started with pid [8782]
started core service [/rosout]
process[player-2]: started with pid [8789]
process[sensor_setup_node-3]: started with pid [8790]
process[my_rviz-4]: started with pid [8796]
[ INFO] [1587748128.983994410]: scenario 0002
[ INFO] [1587748128.985153621]: lidar_height -1.73
[ INFO] [1587748128.985239620]: lidar_z_min -2.4
[ INFO] [1587748128.985358354]: grid_range_min 3
[ INFO] [1587748128.985464895]: grid_range_max 83
[ INFO] [1587748128.985544179]: grid_height 332
[ INFO] [1587748128.985638937]: grid_width 664
[ INFO] [1587748128.985742055]: grid_cell_size 0.25
[ INFO] [1587748128.985845873]: grid_cell_height 0.15
[ INFO] [1587748128.985886009]: grid_bins 470
[ INFO] [1587748128.985981191]: grid_segments 240
[ INFO] [1587748128.986083141]: ransac_tolerance 0.2
[ INFO] [1587748128.986153458]: ransac_iterations 50
[ INFO] [1587748128.986252019]: inv_angular_res 152.789
[ INFO] [1587748128.986307932]: inv_radial_res 4
[player-2] process has finished cleanly
log file: /home/user/.ros/log/42b8fb0c-864e-11ea-92dd-240a641f2dc5/player-2*.log

What steps I have made wrong?

我想向您请教一些问题

我想向您请教一些问题，如果您方便的话是否可以加以下联系方式。我的邮箱[email protected],很希望能收到您的回复。真心感谢。

Warning: No Image Received

I'm using the dataset 2011_09_26 (0001).
RViz opens but there is no output. Map and the 3 images come with the warnings-
No Map Received
No Map Received

What could be the reason?
Also, only the path and the filename had to be taken care of in the launch file right?

Launching sensor processing sensor_launch error

Simon,

Where should I store the directory for dataset_60 result from your GDrive? I know it needs to be renamed as 0060. Everytime I try to do launch, it always gives me error

roslaunch: error: The following input files do not exist: 0060

missing some header files

In helper package, it looks like that there are some missing header files like #include "helper/Object.h" and namespace helper itself seem not to exist in the source code.

sensor_processing package miss key image copy operation

Hi, your project is very awesome, and I am very inspired by this project.
I found the processImage function in sensor_processing package don't work well as show as below:

cv::Mat sem_edge_img, sem_dil_img, sem_output;
	if(params_.sem_ed){
		cv::Canny(sem_image_, sem_edge_img, params_.sem_ed_min,
			params_.sem_ed_max, params_.sem_ed_kernel);
		cv::dilate(sem_edge_img, sem_dil_img, cv::Mat(), 
			cv::Point(-1, -1), 1, 1, 1);
		sem_image_.copyTo(sem_output, sem_dil_img);
	}

	// Publish
	cv_bridge::CvImage cv_semantic_image;
	cv_semantic_image.image = sem_output;

when the if(params_.sem_ed) condition is not true, the sem_output is null image, I think your should add sem_output = sem_image_ above cv_bridge::CvImage cv_semantic_image;.
Thanks!

Empty semantic point cloud!

The drive I provided from readme 0012 is downloaded from goose's address Zip, which only contains 0012 Bag file. After running roslaunch, the following results appear:
[ WARN] [1646748554.890493462]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748554.894613255]: Empty semantic point cloud!
[ WARN] [1646748555.400600677]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748555.404701194]: Empty semantic point cloud!
[ WARN] [1646748555.923053514]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748555.927108425]: Empty semantic point cloud!
[ WARN] [1646748556.439107921]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748556.443300419]: Empty semantic point cloud!
[ WARN] [1646748556.961663497]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748556.965640884]: Empty semantic point cloud!
[ WARN] [1646748557.478409474]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748557.482468366]: Empty semantic point cloud!
[ WARN] [1646748557.997741672]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748558.001720665]: Empty semantic point cloud!
[ WARN] [1646748558.511803933]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748558.515771263]: Empty semantic point cloud!
[ WARN] [1646748559.034038873]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748559.038129333]: Empty semantic point cloud!
[ WARN] [1646748559.544194778]: Hardcode path in sensor_fusion.cpp processImage()!
[ WARN] [1646748559.548134172]: Empty semantic point cloud!

Do you have any other files to download, or do I need to do some other work? Please help me list a detailed reproduction process, the files I need to download and the work I need to complete. I'm just beginning to learn this field. Thank you very much for your help

Segment personal image in ros

Hi @appinho , Thanks for your sharing this great work,
I have a image which contains trees, buildings and sky. How can i use your package to segment my image ?
For example, segment the trees, buildings and sky area
thanks a lot,
Best,
Welson

What do these metrics ‘MOTAL‘， ’MODA‘ and ‘MODP‘ mean?

I guess 'MODA' means 'multi-object detection accuracy' and 'MODP' means 'multi-object detection precision' ? What about the 'MOTAL'?

Is there any easy way to visualize the bounding box?

Hello. I am new to this, and I have run all other .launch files given. I have seen the PointClouds successfully, but I cannot generate the bounding box shown in the Readme gif.
Is there any easy way to visualize the bounding box, or am I making a simple mistake? (I think may be it is related to the visualization module?)
Thanks!

How to run your code?

As title.

undefined reference to `log@GLIBC_2.29'

when I catkin_make the project, The problem about ros shows:
/usr/bin/ld: /lib/x86_64-linux-gnu/libicui18n.so.66: undefined reference to log@GLIBC_2.29'
/usr/bin/ld: /lib/x86_64-linux-gnu/libicui18n.so.66: undefined reference to pow@GLIBC_2.29' collect2: error: ld returned 1 exit status make[2]: *** [SARosPerceptionKitti/evaluation/CMakeFiles/evaluation.dir/build.make:251：/home/seu/data_backup/new_disk/bbdata/catkin_ws/devel/lib/evaluation/evaluation] 错误 1 make[1]: *** [CMakeFiles/Makefile2:3444：SARosPerceptionKitti/evaluation/CMakeFiles/evaluation.dir/all] 错误 2
would you help me solve this problem?

Segmented Images registration

How do I visualize the scenario as shown in
https://raw.githubusercontent.com/appinho/ROS_Perception_Package_Kitti_Dataset/master/videos/rviz.gif
That is, segmented images registered on the point cloud ?