Here are different things you can try to help me solve out the problem:

• first can you tell me if it crashes always at the same frame?
• can you try to run the code by disabling yolo in the launch file (set use_yolo to false)
• if disabling yolo doesn't make any difference can you try with disabling the moving object detection (set enable_mod to false in the launch file)
• if it is still not working can you try to rebuild the package setting CMAKE_BUILD_TYPE to Debug and do proper debugging using gdb (http://wiki.ros.org/roslaunch/Tutorials/Roslaunch%20Nodes%20in%20Valgrind%20or%20GDB), to at least see exactly where the program crashes (if the problem is on the device side (gpu) we will have to use cuda debbuging tool after that)

from supersurfel_fusion.

@BruceCanovas Hi,I ran successfully with disabling the moving object detection (set enable_mod to false in the launch file),and it seems to be a problem with moving object detection module.
After disabling the moving object detection module, the node appears to run stably.
Here is the log output,and it looks like the end of the sequence caused the node to stop.

Sparse VO success
ICP success
bestKey : 369
Total time = 22 ms
nb supersurfels: 11231
stamp: 1017
Max model size = 1.1515 MB
Mean runtime = 16.8075

CachedAllocator::free_all(): cleaning up after ourselves...
================================================================================REQUIRED process [supersurfel_fusion_rgbd_benchmark_node-2] has died!
process has finished cleanly
log file: /home/robocup3d/.ros/log/491cd8cc-3d16-11eb-8208-0492264b1059/supersurfel_fusion_rgbd_benchmark_node-2*.log
Initiating shutdown!

[rqt_reconfigure-4] killing on exit
[rviz-3] killing on exit
[supersurfel_fusion_rgbd_benchmark_node-2] killing on exit
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

Because of time, I haven't done a single step debugging yet.

from supersurfel_fusion.

And when using moving object detection without Yolo, does it fail?

from supersurfel_fusion.

yes,it will fail when using moving object detection without Yolo

from supersurfel_fusion.

@BruceCanovas I'm sorry to bother you again, I just debug the node in gdb, and here is the error output.
Is it a problem about opencv_version or an Opencv program error caused by the odometer not being initialized ?
Thank you very much!

[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作] (An inappropriate ioctl operation on a device)
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[New Thread 0x7fff5af18700 (LWP 27124)]
[New Thread 0x7fff5a717700 (LWP 27125)]
[New Thread 0x7fff59f16700 (LWP 27126)]
[New Thread 0x7fff59715700 (LWP 27127)]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[New Thread 0x7fff45fff700 (LWP 27134)]
[New Thread 0x7fff457fe700 (LWP 27135)]
[New Thread 0x7fff44efb700 (LWP 27136)]
[tcsetpgrp failed in terminal_inferior: 对设备不适当的 ioctl 操作]
[New Thread 0x7fff41dff700 (LWP 27151)]
[New Thread 0x7fff419fe700 (LWP 27153)]
[New Thread 0x7fff415fd700 (LWP 27152)]
[New Thread 0x7fff411fc700 (LWP 27154)]
[New Thread 0x7fff409fa700 (LWP 27155)]
[New Thread 0x7fff405f9700 (LWP 27156)]
[New Thread 0x7fff40dfb700 (LWP 27157)]
[New Thread 0x7fff07fff700 (LWP 27158)]
[Thread 0x7fff07fff700 (LWP 27158) exited]
bestKey : 0
Total time = 40 ms
nb supersurfels: 1200
stamp: 0
Max model size = 0.119019 MB
Mean runtime = 40

[New Thread 0x7fff07fff700 (LWP 27159)]
[Thread 0x7fff07fff700 (LWP 27159) exited]
[New Thread 0x7fff07fff700 (LWP 27160)]
[Thread 0x7fff07fff700 (LWP 27160) exited]
Sparse VO success
ICP success
bestKey : 0
Total time = 28 ms
nb supersurfels: 566
stamp: 1
Max model size = 0.119019 MB
Mean runtime = 34

[New Thread 0x7fff07fff700 (LWP 27161)]
[Thread 0x7fff07fff700 (LWP 27161) exited]
[New Thread 0x7fff07fff700 (LWP 27162)]
[Thread 0x7fff07fff700 (LWP 27162) exited]

Thread 1 "supersurfel_fus" received signal SIGSEGV, Segmentation fault.
0x00007fffec1afd8c in cv::optflow::DISOpticalFlowImpl::PatchInverseSearch_ParBody::operator()(cv::Range const&) const ()
from /usr/local/lib/libopencv_optflow.so.3.4

from supersurfel_fusion.

look like a problem with DISOpticalFlow

from supersurfel_fusion.

Ok, could you compile in Debug mode (with the flag -g) and show me the backtrace on gdb please? About OpenCV, do you build with the contrib modules? Because the DIS optical flow is in the contribs. I will try to see if it's related to my code or to OpenCV. In the meantime you can only rely on YOLOv4 to filter out possibly moving object.

from supersurfel_fusion.

@BruceCanovas hi,sorry to bother you again.Now I find the error code and there are

    /***** Compute dense optical flow *****/
    dof->calc(gray, gray_estimate, optflow);

of function void MotionDetection::detectMotionSimple in the supersurfel_fusion/core/src/motion_detection.cu

Now it looks like a simple call to one of OpenCV's functions,but I should have been build opencv with the contrib modules successfully.But I will verify if the compilation is successful.

from supersurfel_fusion.

Maybe this can help: opencv/opencv_contrib#1476

from supersurfel_fusion.

I have tried all the solutions including the one you recommended. But it do not work out finally, I do have the libopencv_optflow.so.3.4.
Maybe here is a problem with my computer, I give it up and I am going to write my own code to detect moving objects without using optical streams. Anyway, thank you so much!

from supersurfel_fusion.

Ok, I am currently thinking on proposing an alternative more accurate than the DIS optical flow. In the source code, in "core/src/motion_detection.cu", there is a method called detectMotionYoloOnly () that you can use to perform MOD only with YOLO. Some changes may be done to make it more accurate :

• for the moment it will only detect as dynamic "object" labeled as "human", but it can easiy be extended to other classes such as dog, cat, cars etc ...
• to extract the superpixels inside the detected bounding boxes that belong to the human, I perform simple flood filling segmentation on the depth using the bounding box center as initial seed (which may fail if the center is a foreground point). A more elaborate method to select the initial seed should provide more robust results. Otherwise I yearn to use grabcut or simple gmm on superpixels data but don't have the time to make it work for the moment.
• a last and easy solution is to get rid of all the superpixels inside the detect bounding boxes but you may lose a lot of "static elements" and the camera pose tracking might fail if there's not enough data

from supersurfel_fusion.