jderobot / sdslam Goto Github PK

View Code? Open in Web Editor NEW

This project forked from raulmur/orb_slam2

47.0 12.0 19.0 42.87 MB

Semi Direct SLAM (SD-SLAM) for Monocular and RGB-D Cameras

License: Other

CMake 8.27% Shell 0.10% C++ 91.26% C 0.37%

sdslam's Introduction

SD-SLAM

Authors: Eduardo Perdices

SD-SLAM is a real-time SLAM library for Monocular and RGB-D cameras that computes the camera trajectory and a sparse 3D reconstruction (in the RGB-D case with true scale). It is able to detect loops and relocalize the camera in real time.

1. License

SD-SLAM is released under a GPLv3 license.

2. Prerequisites

We have tested the library in Ubuntu 16.04, but it should be easy to compile in other platforms.

Pangolin

We use Pangolin for visualization and user interface. Dowload and install instructions can be found at: https://github.com/stevenlovegrove/Pangolin.

OpenCV

We use OpenCV to manipulate images and features. Dowload and install instructions can be found at: http://opencv.org. Required at leat 2.4.3. Tested with OpenCV 2.4.11 and OpenCV 3.2.

Eigen3

We use Eigen3 to perform matrices operations. Download and install instructions can be found at: http://eigen.tuxfamily.org. Required at least 3.1.0.

g2o (Included in extra folder)

We use modified versions of the g2o library to perform non-linear optimizations. This library (which is BSD) is included in the extra folder.

3. Building SD-SLAM library and examples

Clone the repository:

git clone https://github.com/eperdices/SD-SLAM

We provide a script build.sh to build SD-SLAM. Please make sure you have installed all required dependencies (see section 2). Execute:

cd SD-SLAM
chmod +x build.sh
./build.sh

This will create libSD_SLAM.so at lib folder and the executables monocular and rgbd in Examples folder.

4. Monocular Examples

Inside PATH_TO_SEQUENCE_FOLDER there must be a file named files.txt with each image filename.

./Examples/Monocular/monocular Examples/Monocular/X.yaml PATH_TO_SEQUENCE_FOLDER

TUM Dataset

Download a sequence from http://vision.in.tum.de/data/datasets/rgbd-dataset/download and uncompress it.
Change X.yaml to TUM1.yaml,TUM2.yaml or TUM3.yaml for freiburg1, freiburg2 and freiburg3 sequences respectively.

EuRoC Dataset

Download a sequence (ASL format) from http://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets
Change X.yaml to EuRoC.yaml.

Live Camera

You can use a USB camera to run SD-SLAM with live images. Change N to the proper /dev/videoN value where your camera is connected.

./Examples/Monocular/monocular Examples/Monocular/X.yaml N

Initialization with pattern

A pattern can be emploid to create the initial map using real scale. The pattern detected is the same chessboard used for camera calibration, located inside Examples/Calibration folder.

To use a pattern, set UsePattern: 1 in yaml configuration file.

5. RGB-D Example

TUM Dataset

Download a sequence from http://vision.in.tum.de/data/datasets/rgbd-dataset/download and uncompress it.
Associate RGB images and depth images using the python script associate.py located inside Examples/RGB-D folder and executing:

python Examples/RGB-D/associate.py PATH_TO_SEQUENCE/rgb.txt PATH_TO_SEQUENCE/depth.txt > associations.txt

Execute the following command. Change TUMX.yaml to TUM1.yaml,TUM2.yaml or TUM3.yaml for freiburg1, freiburg2 and freiburg3 sequences respectively. Change PATH_TO_SEQUENCE_FOLDERto the uncompressed sequence folder. Change ASSOCIATIONS_FILE to the path to the corresponding associations file.

./Examples/RGB-D/rgbd Examples/RGB-D/TUMX.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE

6. Monocular and IMU Fusion Example

Inside PATH_TO_SEQUENCE_FOLDER there must be a file named ''files.txt'' with each image filename. IMU data has to be stored in a csv file where each line has a timestamp and 6 measurements (3 from gyroscope and 3 from accelerometer).

./Examples/Fusion/monocular_imu Examples/Monocular/X.yaml PATH_TO_SEQUENCE_FOLDER IMU_DATA_FILE.csv

EuRoC Dataset

Download a sequence (ASL format) from http://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets
Change X.yaml to EuRoC.yaml.

7. Processing your own sequences

You will need to create a settings file with the calibration of your camera. See the settings file provided for the TUM dataset for monocular or RGB-D cameras. We use the calibration model of OpenCV. See the examples to learn how to create a program that makes use of the SD-SLAM library and how to pass images to the SLAM system. RGB-D input must be synchronized and depth registered.

Camera Calibration

We provide a tool to calibrate your camera using the calibration model of OpenCV. You need to print the pattern found inside Examples/Calibration folder and take several pictures of this pattern from different points of view. Then store these pictures in a folder and execute the calibration tool.

./Examples/Calibration/calibration PATH_TO_IMAGES_FOLDER

Inside PATH_TO_IMAGES_FOLDER there must be a file named ''files.txt'' with each image filename.

You can check if the intrinsic parameters calculated are accurate checking the rectified images stored in PATH_TO_IMAGES_FOLDER.

8. ROS Examples

Building the node

Add the path including Examples/ROS/SD-SLAM to the ROS_PACKAGE_PATH environment variable. Replace PATH by the folder where you cloned SD-SLAM:

echo "export ROS_PACKAGE_PATH=\$ROS_PACKAGE_PATH:PATH/SD-SLAM/Examples/ROS" >> ~/.bashrc

Execute build_ros.sh script:

chmod +x build_ros.sh
./build_ros.sh

Running Monocular Node

Monocular node reads RGB images from topic /camera/rgb/image_raw. You will need to provide a settings file (see examples above).

rosrun SD-SLAM Monocular Examples/ROS/SD-SLAM/ROS.yaml

Running RGBD Node

RGBD node reads RGB images from topic /camera/rgb/image_raw and depth images from topic /camera/depth/image_raw. You will need to provide a settings file (see examples above).

rosrun SD-SLAM RGBD Examples/ROS/SD-SLAM/ROS.yaml

Running Fusion Node

Fusion node reads RGB images from topic /camera/rgb/image_raw and IMU data from topic /imu_data. You will need to provide a settings file (see examples above).

rosrun SD-SLAM Fusion Examples/ROS/SD-SLAM/ROS.yaml

Running RGBD with roslaunch

Several launch files has been provided so the node RGBD can be run with those. In order to run SD-SLAM with the configuration for TUM freidburg1 sequences run

roslaunch SD-SLAM sdslam_TUM1.launch

In order to run SD-SLAM with the configuration for TUM freidburg1 sequences and odometry_evaluation_file_creation to create a groundtruth file that can be compare with the groundtruths of TUM

roslaunch SD-SLAM sdslam_TUM1_evaluation_file.launch

9. Reading saved data

Mapping data can be stored in a YAML file. You can save the current map at any moment pressing the Stop and Save button, or it will be created automatically when a sequence is completed.

Saved data can be loaded afterwards both in monocular and RGBD modes:

9.1 Monocular Example

Add a new parameter at the end of the standard monocular command. Change PATH_TO_SAVED_MAP to the path to the corresponding YAML file.

./Examples/Monocular/monocular Examples/Monocular/X.yaml PATH_TO_SEQUENCE_FOLDER PATH_TO_SAVED_MAP

9.2 RGBD Example

Add a new parameter at the end of the standard RGBD command. Change PATH_TO_SAVED_MAP to the path to the corresponding YAML file.

./Examples/RGB-D/rgbd Examples/RGB-D/X.yaml PATH_TO_SEQUENCE_FOLDER ASSOCIATIONS_FILE PATH_TO_SAVED_MAP

9.3 ROS Monocular Node Example

Add a new parameter at the end of the standard ROS monocular command. Change PATH_TO_SAVED_MAP to the path to the corresponding YAML file.

rosrun SD-SLAM Monocular Examples/ROS/SD-SLAM/ROS.yaml PATH_TO_SAVED_MAP

9.4 ROS RGBD Node Example

Add a new parameter at the end of the standard ROS RGBD command. Change PATH_TO_SAVED_MAP to the path to the corresponding YAML file.

rosrun SD-SLAM RGBD Examples/ROS/SD-SLAM/ROS.yaml PATH_TO_SAVED_MAP

10. Localization Mode

You can change between the Default Mode and Localization mode using the GUI of the map viewer.

Default Mode

The system runs in parallal three threads: Tracking, Local Mapping and Loop Closing. The system localizes the camera, builds a new map and tries to close loops.

Localization Mode

This mode can be used when you have a good map of your working area. In this mode the Local Mapping and Loop Closing are deactivated. The system localizes the camera in the map (which is no longer updated), using relocalization if needed.

11. Android Compilation

You can create a SD-SLAM library for Android running:

chmod +x build_android.sh
./build_android.sh

Before running this script, read https://github.com/taka-no-me/android-cmake to configure your workspace properly.

sdslam's People

Contributors

Stargazers

Watchers

Forkers

eperdices gvc0461082002 hddid fjpalfer javimdr fangzheng81 esperanzadiaz311 dmchen2015 omar-ogm gunjangiri elekezem plushpluto smartechmt orcilano pabloasensio fengzhang123 bdilman cupido2 sfaster

sdslam's Issues

Weird behaviour of RGDB with Realsense D435 data in ROS

Hi,

I've been testing both mono and RGBD SDSLAM with data from a new Intel Realsense D435 RGBD camera, using ROS. While mono-SLAM goes relatively well, the RGBD version gives a weird output, throwing huge pose values. Please have a look at it in this video (0:00 mono, 1:15 RGBD) .

Rosbags were recorded at 1280x720, 30 FPS. The D435 was previously calibrated, and the following image topics were added to the .yaml file:

RGB image raw topic : /camera/color/image_raw
Registered depth image raw topic: /camera/aligned_depth_to_color/image_raw

If you need to have a look at the DepthCloud, there is a video here. Also, the original rosbag is available for download here.

I've recorded some other rosbags and the behaviour is the same. If you need any of them, please let me know.

some issues

Hello, is it a complete project?Does it have paper?Emmm, and I want to know how do you finish loop detection. Thank you very much!

Stereo Camera?

What would need to be done to use a stereo camera with SDslam? Thanks!

[Feature Request] Make SD-SLAM work on non texture environments using only visual depth information

Allow SD-SLAM when working on RGBD mode to use the depth information to keep tracking even if there is no texture at all.
For this an odometry depth only based method is needed.

loop detetion

I see the project doesn't have DBOW files, it doesn't have loop detection and optimization, right?

Improve accuracy using IMU

Current RGB-IMU Fusion doesn't improve Monocular SLAM accuracy. It should be reviewed to extract more orientation information from IMU data.

Trying to use pattern initialization.

Hi,

I’m working with SD-SLAM in an augmented reality app for Android and i had some scale issues, so i tried to use the pattern initialization.

From what I have found, to use this initialization you only have to change the variable kUsePattern_ that is in the Config class. This is one of the variables that are loaded from the specified file when using the ReadParameters method of that same class.

However, as I’m programming on mobile, I think it can be an improvement to have a public method to access this value.

Also, currently you can’t access the value of the size of cells, so when printing the pattern they are probably different from the values in the code. Being able to access these values would help to use this type of initialization.

Use IMU measurements

Use IMU as input sensor to improve localization when it is available

[sdslam+_depth] Build.sh script fails due to mrpt-poses

Hi @omar-ogm

I'm trying to build the sdslam+_depth branch but I'm getting a CMake error related to "mrpt-poses". This is the complete build output:

diego@JOULE:~/JDEROBOT/SD-SLAM+/SDslam$ sh build.sh
Configuring and building SD_SLAM ...
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
Build type: Release
-- Compiling on Unix
-- Using CATKIN_DEVEL_PREFIX: /home/diego/JDEROBOT/SD-SLAM+/SDslam/build/devel
-- Using CMAKE_PREFIX_PATH: /home/diego/catkin_ws/devel;/opt/ros/melodic;/opt/jderobot;/opt/jderobot
-- This workspace overlays: /home/diego/catkin_ws/devel;/opt/ros/melodic
-- Found PythonInterp: /usr/bin/python2 (found suitable version "2.7.17", minimum required is "2") 
-- Using PYTHON_EXECUTABLE: /usr/bin/python2
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/diego/JDEROBOT/SD-SLAM+/SDslam/build/test_results
-- Found gtest sources under '/usr/src/googletest': gtests will be built
-- Found gmock sources under '/usr/src/googletest': gmock will be built
-- Found PythonInterp: /usr/bin/python2 (found version "2.7.17") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.23
-- BUILD_SHARED_LIBS is on
-- Using these message generators: gencpp;geneus;genlisp;gennodejs;genpy
CMake Error at /usr/share/cmake-3.10/Modules/CMakeFindDependencyMacro.cmake:48 (find_package):
  By not providing "Findmrpt-poses.cmake" in CMAKE_MODULE_PATH this project
  has asked CMake to find a package configuration file provided by
  "mrpt-poses", but CMake did not find one.

  Could not find a package configuration file provided by "mrpt-poses" with
  any of the following names:

    mrpt-posesConfig.cmake
    mrpt-poses-config.cmake

  Add the installation prefix of "mrpt-poses" to CMAKE_PREFIX_PATH or set
  "mrpt-poses_DIR" to a directory containing one of the above files.  If
  "mrpt-poses" provides a separate development package or SDK, be sure it has
  been installed.
Call Stack (most recent call first):
  /usr/share/mrpt/mrpt-config.cmake:26 (find_dependency)
  CMakeLists.txt:64 (find_package)


-- Configuring incomplete, errors occurred!
See also "/home/diego/JDEROBOT/SD-SLAM+/SDslam/build/CMakeFiles/CMakeOutput.log".
See also "/home/diego/JDEROBOT/SD-SLAM+/SDslam/build/CMakeFiles/CMakeError.log".
make: *** No se especificó ningún objetivo y no se encontró ningún makefile.  Alto.

However, I check if mrpt-poses is installed with apt list --installed | grep mrpt-poses and everything seems ok:

libmrpt-poses2.0/bionic,now 1:2.0.4~snapshot20200525-1143-git-7ea9b7e8-bionic-1 amd64 [instalado, automático]

Error while compiling LoopClosing.cc

In file included from /usr/include/c++/9/map:61,
                 from /home/pablo/Desktop/master/SDslam/src/KeyFrame.h:28,
                 from /home/pablo/Desktop/master/SDslam/src/LoopClosing.h:30,
                 from /home/pablo/Desktop/master/SDslam/src/LoopClosing.cc:25:
/usr/include/c++/9/bits/stl_map.h: In instantiation of ‘class std::map<SD_SLAM::KeyFrame*, g2o::Sim3, std::less<SD_SLAM::KeyFrame*>, Eigen::aligned_allocator<std::pair<const SD_SLAM::KeyFrame*, g2o::Sim3> > >’:
/home/pablo/Desktop/master/SDslam/src/LoopClosing.cc:402:19:   required from here
/usr/include/c++/9/bits/stl_map.h:122:71: error: static assertion failed: std::map must have the same value_type as its allocator
  122 |       static_assert(is_same<typename _Alloc::value_type, value_type>::value,
      |                                                                       ^~~~~
make[2]: *** [CMakeFiles/SD_SLAM.dir/build.make:115: CMakeFiles/SD_SLAM.dir/src/LoopClosing.cc.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:192: CMakeFiles/SD_SLAM.dir/all] Error 2
make: *** [Makefile:130: all] Error 2

Error when launching monocular with TUM dataset

Hi,

I've just installed the tool and I'm trying to run the "freiburg1_desk" example from TUM dataset using monocular and TUM1.yaml. However, I'm getting an error, reproduced below. Any ideas about what could be happening?

I've renamed the rgb.txt file to files.txt and moved it into the /rgb folder.

diego@EINSTEIN:~/SD-SLAM$ ./Examples/Monocular/monocular Examples/Monocular/TUM1.yaml DIEGO_EJEMPLOS/rgbd_dataset_freiburg1_desk/rgb
[INFO] Sequence has 616 images
[DEBUG] Input sensor was set to Monocular

Camera Parameters:

fx: 517.306
fy: 516.469
cx: 318.643
cy: 255.314
k1: 0.262383
k2: -0.953104
k3: 1.16331
p1: -0.005358
p2: 0.002628
fps: 30

ORB Extractor Parameters:

Number of Features: 1000
Scale Levels: 5
Scale Factor: 2
Fast Threshold: 20
[DEBUG] Loop closing activated
[INFO] Reading Frame DIEGO_EJEMPLOS/rgbd_dataset_freiburg1_desk/rgb/#

[ERROR] Failed to load image at: DIEGO_EJEMPLOS/rgbd_dataset_freiburg1_desk/rgb/#
[xcb] Unknown sequence number while appending request
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
monocular: ../../src/xcb_io.c:161: append_pending_request: Assertion !xcb_xlib_unknown_seq_number' failed. Abortado (core' generado)

Thanx in advance,
Diego.

[Feature Request] ROSify pose output

Hi again,

It seems that in the present ROS version of the algorithm, the nodes get data from ROS topics, but the output is not published as a ROS topic using the standard geometry_msgs/Pose message.

It would be nice to have the pose ROSified, to allow communication with other ROS nodes.

Thanx.

diego@EINSTEIN:~/SD-SLAM$ ./build_ros.sh
Ruilding ROS nodes
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python (found version "2.7.12") 
[rosbuild] Building package SD-SLAM
[rosbuild] Cached build flags older than manifests; calling rospack to get flags
-- Using CATKIN_DEVEL_PREFIX: /home/diego/SD-SLAM/Examples/ROS/SD-SLAM/build/devel
-- Using CMAKE_PREFIX_PATH: /home/diego/catkin_ws/devel;/opt/ros/kinetic
-- This workspace overlays: /home/diego/catkin_ws/devel;/opt/ros/kinetic
-- Using PYTHON_EXECUTABLE: /usr/bin/python
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Skip enable_testing() for dry packages
-- Using CATKIN_TEST_RESULTS_DIR: /home/diego/SD-SLAM/Examples/ROS/SD-SLAM/build/test_results
-- Found gmock sources under '/usr/src/gmock': gmock will be built
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found gtest sources under '/usr/src/gmock': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.18
-- BUILD_SHARED_LIBS is on
[rosbuild] using multiarch 'x86_64-linux-gnu' for finding Boost
Build type: Release
-- Found Eigen3: /usr/local/include/eigen3 (Required is at least version "3.1.0") 
-- Boost version: 1.58.0
-- Found the following Boost libraries:
--   system
-- Configuring done
-- Generating done
-- Build files have been written to: /home/diego/SD-SLAM/Examples/ROS/SD-SLAM/build
Scanning dependencies of target rospack_genmsg_libexe
[  0%] Built target rospack_genmsg_libexe
Scanning dependencies of target rosbuild_precompile
[  0%] Built target rosbuild_precompile
Scanning dependencies of target Fusion
Scanning dependencies of target RGBD
Scanning dependencies of target Monocular
make[2]: *** No hay ninguna regla para construir el objetivo '../../../../lib/libSD_SLAM.so', necesario para '../RGBD'.  Alto.
make[2]: *** No hay ninguna regla para construir el objetivo '../../../../lib/libSD_SLAM.so', necesario para '../Monocular'.  Alto.
make[2]: *** Se espera a que terminen otras tareas....
make[2]: *** Se espera a que terminen otras tareas....
make[2]: *** No hay ninguna regla para construir el objetivo '../../../../lib/libSD_SLAM.so', necesario para '../Fusion'.  Alto.
make[2]: *** Se espera a que terminen otras tareas....
[ 16%] Building CXX object CMakeFiles/Monocular.dir/src/ros_monocular.cc.o
[ 50%] Building CXX object CMakeFiles/RGBD.dir/src/ros_rgbd.cc.o
[ 50%] Building CXX object CMakeFiles/Fusion.dir/src/ros_fusion.cc.o
CMakeFiles/Makefile2:515: fallo en las instrucciones para el objetivo 'CMakeFiles/Monocular.dir/all'
make[1]: *** [CMakeFiles/Monocular.dir/all] Error 2
make[1]: *** Se espera a que terminen otras tareas....
CMakeFiles/Makefile2:584: fallo en las instrucciones para el objetivo 'CMakeFiles/RGBD.dir/all'
make[1]: *** [CMakeFiles/RGBD.dir/all] Error 2
CMakeFiles/Makefile2:621: fallo en las instrucciones para el objetivo 'CMakeFiles/Fusion.dir/all'
make[1]: *** [CMakeFiles/Fusion.dir/all] Error 2
Makefile:127: fallo en las instrucciones para el objetivo 'all'
make: *** [all] Error 2

Update OpenCV to OpenCV2 in Vio-sdslam branch

I guess this will make installation easier as well as update some opencv functions.

jderobot / sdslam Goto Github PK

sdslam's Introduction

SD-SLAM

1. License

2. Prerequisites

Pangolin

OpenCV

Eigen3

g2o (Included in extra folder)

3. Building SD-SLAM library and examples

4. Monocular Examples

TUM Dataset

EuRoC Dataset

Live Camera

Initialization with pattern

5. RGB-D Example

TUM Dataset

6. Monocular and IMU Fusion Example

EuRoC Dataset

7. Processing your own sequences

Camera Calibration

8. ROS Examples

Building the node

Running Monocular Node

Running RGBD Node

Running Fusion Node

Running RGBD with roslaunch

9. Reading saved data

9.1 Monocular Example

9.2 RGBD Example

9.3 ROS Monocular Node Example

9.4 ROS RGBD Node Example

10. Localization Mode

Default Mode

Localization Mode

11. Android Compilation

sdslam's People

Contributors

Stargazers

Watchers

Forkers

sdslam's Issues

Recommend Projects

Recommend Topics

Recommend Org