Can we use the GPU to run the code? about dynaslam HOT 17 CLOSED

And another problem is when I use it on the rgbd_dataset_freiburg3_walking_rpy dataset. It always show that

Light Tracking not working because Tracking is not initialized...

another problem is that it crashed with

OpenCV Error: Assertion failed (type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2)) in gemm, file /home/robotics/Downloads/opencv-2.4.11/modules/core/src/matmul.cpp, line 711
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/robotics/Downloads/opencv-2.4.11/modules/core/src/matmul.cpp:711: error: (-215) type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2) in function gemm
` this fault, have you ever met ?

from dynaslam.

I'm sorry to bothering you. In my computer, I figure out that the tensorflow is using the CPU to run the code. How to make it use the GPU？

I will take a look and let you know asap.

from dynaslam.

And another problem is when I use it on the rgbd_dataset_freiburg3_walking_rpy dataset. It always show that

Light Tracking not working because Tracking is not initialized...

another problem is that it crashed with

OpenCV Error: Assertion failed (type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2)) in gemm, file /home/robotics/Downloads/opencv-2.4.11/modules/core/src/matmul.cpp, line 711
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/robotics/Downloads/opencv-2.4.11/modules/core/src/matmul.cpp:711: error: (-215) type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2) in function gemm
` this fault, have you ever met ?

Are you using monocular or RGB-D mode? I will try to reproduce the problem.

from dynaslam.

Yes, The dataset is the TUM Dynamic about waking. The tracking is always failed to get initialized, but can work in other dataset. I don't know why.

from dynaslam.

I am not sure if you are using the RGB-D or monocular setup. In any case:

1. If you are running RGB-D DynaSLAM. I am running this command to try to reproduce the error you have encountered:

./Examples/RGB-D/rgbd_tum Vocabulary/ORBvoc.txt Examples/RGB-D/TUM3.yaml /my/path/to/rgbd_dataset_freiburg3_walking_rpy Examples/RGB-D/associations/fr3_walking_rpy.txt /my/output/path/for/masks

It sometimes takes a few frames to initialize but it usually succeeds. As long as the system is not initialized it will show Light Tracking not working because Tracking is not initialized...

2. If you are running the system with the monocular setup, I am running the following command:

./Examples/Monocular/mono_tum Vocabulary/ORBvoc.txt Examples/Monocular/TUM3.yaml /my/path/to/rgbd_dataset_freiburg3_walking_rpy /my/output/path/for/masks

In this case it is harder to initialize the system with this sequence. This sequence is highly dynamic and therefore it sometimes fails to initialize because there are few keypoints. What you could do is to increase the number of extracted features per frame in the corresponding settings file TUM3.yaml from 1000 to 1500 for example (ORBextractor.nFeatures: 1500).

from dynaslam.

OK, I'll try it again tonight

from dynaslam.

However, I have not been able to reproduce this other error:

OpenCV Error: Assertion failed (type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2)) in gemm, file /home/robotics/Downloads/opencv-2.4.11/modules/core/src/matmul.cpp, line 711```
terminate called after throwing an instance of 'cv::Exception'
what():  /home/robotics/Downloads/opencv-2.4.11/modules/core/src/matmul.cpp:711: error: (-215) type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2) in function gemm

Let us know if this happens again.

from dynaslam.

OK， My computer is I5, so it is very slow to run the code. I left it run at the lab and to take a sleep, I found this at the next day morning. Maybe I'll try it tonight again to see what's going on

from dynaslam.

Hi, I am having the same error.

OpenCV Error: Assertion failed (type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2)) in gemm, file /home/ri/OpenCV/opencv-2.4.11/modules/core/src/matmul.cpp, line 711
terminate called after throwing an instance of 'cv::Exception'
  what():  /home/ri/OpenCV/opencv-2.4.11/modules/core/src/matmul.cpp:711: error: (-215) type == B.type() && (type == CV_32FC1 || type == CV_64FC1 || type == CV_32FC2 || type == CV_64FC2) in function gem

I am using OpenCV 2.4.11 and I am running this on GPU 1070. Everything appears to be fine initially and I can get correct mask and output on RGB-D sequence. But after a while the system crashes with the above error.

from dynaslam.

ok, I will take a look at it and try to reproduce the error

from dynaslam.

Here's the back trace. It seems the error occur at Geometry::FillRGBD this method in one of the cv::Mat multiplication. Also, I notice that both Gatsby23 and I are using OpenCV 2.4.11 that is not installed in the typical /usr/local/lib but elsewhere. I had OpenCV3 installed already and I don't want to mess up my environment. I wonder if this contributes to anything. Thank you!

Backtrace:

#0  0x00007ffff5e5a428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff5e5c02a in __GI_abort () at abort.c:89
#2  0x00007ffff706384d in __gnu_cxx::__verbose_terminate_handler() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3  0x00007ffff70616b6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4  0x00007ffff7061701 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5  0x00007ffff7061919 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6  0x00007ffff7570c55 in cv::error(cv::Exception const&) () from /home/ri/DynaSLAM/lib/libDynaSLAM.so
#7  0x00007ffff769241a in cv::gemm(cv::_InputArray const&, cv::_InputArray const&, double, cv::_InputArray const&, double, cv::_OutputArray const&, int) ()
   from /home/ri/DynaSLAM/lib/libDynaSLAM.so
#8  0x00007ffff7594bb5 in cv::MatOp_GEMM::assign(cv::MatExpr const&, cv::Mat&, int) const () from /home/ri/DynaSLAM/lib/libDynaSLAM.so
#9  0x00007ffff75678c1 in DynaSLAM::Geometry::FillRGBD(ORB_SLAM2::Frame const&, cv::Mat&, cv::Mat&, cv::Mat&, cv::Mat&) ()
   from /home/ri/DynaSLAM/lib/libDynaSLAM.so
#10 0x00007ffff747db6b in ORB_SLAM2::Tracking::GrabImageRGBD(cv::Mat const&, cv::Mat const&, cv::Mat&, double const&, cv::Mat&, cv::Mat&, cv::Mat&) ()
   from /home/ri/DynaSLAM/lib/libDynaSLAM.so
#11 0x00007ffff74667de in ORB_SLAM2::System::TrackRGBD(cv::Mat const&, cv::Mat const&, cv::Mat&, double const&, cv::Mat&, cv::Mat&, cv::Mat&) ()
   from /home/ri/DynaSLAM/lib/libDynaSLAM.so
#12 0x000000000040900f in main ()

from dynaslam.

@Gatsby23 @dknyxh , hi, i meet some problem, did u fix it??

from dynaslam.

Hi there!

That was actually a bug. Thanks a lot for letting us know.

We changed in the Tracking.cc file in the function cv::Mat Tracking::GrabImageRGBD(const cv::Mat &imRGB,const cv::Mat &imD, cv::Mat &mask, const double &timestamp, cv::Mat &imRGBOut, cv::Mat &imDOut, cv::Mat &maskOut) the following line:

mGeometry.InpaintFrames(mCurrentFrame, mImGray, imDepth, imRGBOut, imMask);

and added the condition:

if (!mCurrentFrame.mTcw.empty())
{
     mGeometry.InpaintFrames(mCurrentFrame, mImGray, imDepth, imRGBOut, imMask);
}

It is already updated in the master branch.

from dynaslam.

Have there been any suggestions on how to move the code to GPU?
I am on python2, and tensorflow 1.3.0.
Will installing tensorflow-gpu 1.3.0 solve it all or is more required?

from dynaslam.

Have there been any suggestions on how to move the code to GPU?
I am on python2, and tensorflow 1.3.0.
Will installing tensorflow-gpu 1.3.0 solve it all or is more required?

I just installed tensorflow-gpu：1.12.3, and it had worked under GPU automatically.

from dynaslam.

Hi there!

That was actually a bug. Thanks a lot for letting us know.

We changed in the Tracking.cc file in the function cv::Mat Tracking::GrabImageRGBD(const cv::Mat &imRGB,const cv::Mat &imD, cv::Mat &mask, const double &timestamp, cv::Mat &imRGBOut, cv::Mat &imDOut, cv::Mat &maskOut) the following line:

mGeometry.InpaintFrames(mCurrentFrame, mImGray, imDepth, imRGBOut, imMask);

and added the condition:

if (!mCurrentFrame.mTcw.empty())
{
     mGeometry.InpaintFrames(mCurrentFrame, mImGray, imDepth, imRGBOut, imMask);
}

It is already updated in the master branch.

Is the condition be used also in function GrabImageRGBD(const cv::Mat &imRGB,const cv::Mat &imD, cv::Mat &mask, const double &timestamp, int ni, int nImages) ? Because the condition is used in GrabImageRGBD(const cv::Mat &imRGB,const cv::Mat &imD, cv::Mat &mask,const double &timestamp, cv::Mat &imRGBOut,cv::Mat &imDOut, cv::Mat &maskOut, int ni,int nImages). The former is used in no_inpaint model, and the latter used in inpaint model.

from dynaslam.

I use NVIDIA 1080 GPU, my env is :cuda 9.0、cudnn 7.5.1、tensorflow 1.12.0、keras 2.2.4、opencv 2.4.11
It can run mono_tum on GPU ok

from dynaslam.