This package implements geometric segmentation of depth clouds based on region growing. First, the incoming clouds are filtered, normals are estimated for each 3D point, and then PCL region growing segmentation is applied.

The rgbd_segmentation package combines the geometric segmentation implemented here with instance-aware semantic segmentation via Mask R-CNN to obtain combined geometric-semantic segmentation results.

This package was developed as part of the TSDF++ framework for multiple dynamic object tracking and reconstruction.

When using this code in your research, please cite the following publication:

Margarita Grinvald, Federico Tombari, Roland Siegwart, and Juan Nieto, TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction, in 2021 IEEE International Conference on Robotics and Automation (ICRA), 2021. [Video]

@INPROCEEDINGS{grinvald2021tsdf,
  author={Grinvald, Margarita and Tombari, Federico and Siegwart, Roland and Nieto, Juan},
  booktitle={2021 IEEE International Conference on Robotics and Automation (ICRA)}, 
  title={{TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction}}, 
  year={2021},
  volume={},
  number={},
  pages={14192-14198},
  doi={10.1109/ICRA48506.2021.9560923}}

The installation has been tested on Ubuntu 16.04 and Ubutnu 20.04.

• ROS
• C++14 for PCL 1.10

Install ROS following the instructions at the ROS installation page. The full install (ros-kinetic-desktop-full, ros-melodic-desktop-full) are recommended.

Make sure to source your ROS setup.bash script by following the instructions on the ROS installation page.

In your terminal, define the installed ROS version and name of the catkin workspace to use:

export ROS_VERSION=kinetic # (Ubuntu 16.04: kinetic, Ubuntu 18.04: melodic)
export CATKIN_WS=~/catkin_ws

If you don't have a catkin workspace yet, create a new one:

mkdir -p $CATKIN_WS/src && cd $CATKIN_WS
catkin init
catkin config --extend /opt/ros/$ROS_VERSION --merge-devel 
catkin config --cmake-args -DCMAKE_CXX_STANDARD=14 -DCMAKE_BUILD_TYPE=Release
wstool init src

Clone the cloud_segmentation repository over HTTPS (no Github account required) and automatically fetch dependencies:

cd $CATKIN_WS/src
git clone https://github.com/ethz-asl/cloud_segmentation.git
wstool merge -t . cloud_segmentation/cloud_segmentation_https.rosinstall
wstool update

Alternatively, clone over SSH (Github account required):

cd $CATKIN_WS/src
git clone [email protected]:ethz-asl/cloud_segmentation.git
wstool merge -t . cloud_segmentation/cloud_segmentation_ssh.rosinstall
wstool update

Build and source the fetched packages:

catkin build
source ../devel/setup.bash # (bash shell: ../devel/setup.bash,  zsh shell: ../devel/setup.zsh)

The code is available under the MIT license.