yankailab / openkai Goto Github PK

Ubuntu 16.04

I`ve tried to follow this guide: http://www.openkai.xyz/x86build.html and got errors while compiling jetson-inference.
jetson-inference/tensorNet.h:159:12: error: ‘Dims3’ in namespace ‘nvinfer1’ does not name a type

Ive tried it with TensorRT 2.1 and 1.0 both. Also ive tried to build OpenKAI and got this: tensorRT.h:3:30: fatal error: cudaMappedMemory.h: No such file or directory.

What`s wrong with this guide?

While building orb_slam2_gpu on my tx2 following errors occur

nvidia@tegra-ubuntu:~/orb_slam2_gpu/build$ make -j2
[ 70%] Built target ORB_SLAM2
[ 73%] Linking CXX executable stereo_kitti
[ 76%] Linking CXX executable rgbd_tum
lib/libORB_SLAM2.so: undefined reference to cv::cuda::copyMakeBorder(cv::_InputArray const&, cv::_OutputArray const&, int, int, int, int, int, cv::Scalar_<double>, cv::cuda::Stream&)' lib/libORB_SLAM2.so: undefined reference to cv::cuda::createGaussianFilter(int, int, cv::Size_, double, double, int, int)'
lib/libORB_SLAM2.so: undefined reference to cv::cuda::resize(cv::_InputArray const&, cv::_OutputArray const&, cv::Size_<int>, double, double, int, cv::cuda::Stream&)' lib/libORB_SLAM2.so: undefined reference to cv::cuda::StreamAccessor::wrapStream(CUstream_st*)'
collect2: error: ld returned 1 exit status
CMakeFiles/stereo_kitti.dir/build.make:127: recipe for target 'stereo_kitti' failed
make[2]: *** [stereo_kitti] Error 1
CMakeFiles/Makefile2:141: recipe for target 'CMakeFiles/stereo_kitti.dir/all' failed
make[1]: *** [CMakeFiles/stereo_kitti.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
lib/libORB_SLAM2.so: undefined reference to cv::cuda::copyMakeBorder(cv::_InputArray const&, cv::_OutputArray const&, int, int, int, int, int, cv::Scalar_<double>, cv::cuda::Stream&)' lib/libORB_SLAM2.so: undefined reference to cv::cuda::createGaussianFilter(int, int, cv::Size_, double, double, int, int)'
lib/libORB_SLAM2.so: undefined reference to cv::cuda::resize(cv::_InputArray const&, cv::_OutputArray const&, cv::Size_<int>, double, double, int, cv::cuda::Stream&)' lib/libORB_SLAM2.so: undefined reference to cv::cuda::StreamAccessor::wrapStream(CUstream_st*)'
collect2: error: ld returned 1 exit status
CMakeFiles/rgbd_tum.dir/build.make:127: recipe for target 'rgbd_tum' failed
make[2]: *** [rgbd_tum] Error 1
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/rgbd_tum.dir/all' failed
make[1]: *** [CMakeFiles/rgbd_tum.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Can you please help?

There's a broken link in the README.md file in the "Build & Run" section.

broken link: https://github.com/yankailab/OpenKAI/blob/master/doc/JetsonTX1/build.md

This was reported by @SteveJos in the ArduPilot/VisionProjects gitter chat: https://gitter.im/ArduPilot/VisionProjects

I followed everything in how to build documentation.

1. Installation of jetson inferrence was just fine
2. Have some issue with /usr/include/opencv2/gpu/gpu.hpp so I had to add following in /src/Base/cv.h
   using namespace std;
   just before #ifdef USE_CUDA
3. Still build error
   /home/nvidia/src/OpenKAI/src/Application/../Config/../SLAM/_ORB_SLAM2.h:13:20: fatal error: System.h: No such file or directory
   I simply cannot find where the System.h file located.
   How can I locate this file correctly?
   Thanks,

Kyu

Even if there are no command-line options, -h and possibly --help should produce help text and terminate the program.

估计是src\Detector目录下的_DetectorBase.h删除造成的吧。
是不是现在不用_DetectorBase这个方法了？

I use your class SegNet.cpp.
Method - void _SegNet::segment(void)
line - Mat segIdx = Mat(m_InputSize.height, m_InputSize.width, CV_32FC1, (void*)begin); Gets an array of the correct structure, but all weekend classes are equal to 0.
Please tell me what could be wrong?

Hi,
I follow OpenKAI/sh/Setup/OpenKAI_dev_setup.sh to make OpenKai on Tx2, bit I got some errors and I don't know where is wrong
My jetpack version is 4.4

I would appreciate it if someone can help
Thanks you!

my error message:

my ccmake environment:

Memory is growing unbounded - so the system OOM killer is killing it.

Suggest running valgrind...

ZED camera is found but not configuring (low-usb-bandwidth error).

1.修改显卡的运算能力 set(CUDA_ARCH "50" CACHE STRING "")
2.SLAM、DARKNET、jetson-inference-batch几个库放在/home/user下要写完整目录，不然会找不到 "cudaMappedMemory.h"、"cudaNormalize.h"、"detectNet.h"、"imageNet.h"等文件。
3. set(OpenKAI_include ${OpenKAI_include} "${ORB_SLAM2_root} ${ORB_SLAM2_root}/include ${Pangolin_INCLUDE_DIRS}") 要分开写，
set(OpenKAI_include ${OpenKAI_include} "${ORB_SLAM2_root} ")
set(OpenKAI_include ${OpenKAI_include} "${ORB_SLAM2_root}/include")
set(OpenKAI_include ${OpenKAI_include} "${Pangolin_INCLUDE_DIRS}")
不然会找不到 {ORB_SLAM2_root}/include目录下的system.h。

hi everyone,

i'm trying to test the functions that are mentioned in this "tutorial": https://discuss.ardupilot.org/t/ardupilot-openkai-zed-for-non-gps-navigation/16308 .

i have a Nvidia Jetson TX2 with Jetpack 4.1 flashed and tried to follow the recommended installation guide here: http://docs.openkai.xyz/tegrabuild.html#optional-install-zed-driver , i know that some of the programs are outdated, but i tried my best to follow the instructions and to get the actual versions of the programs. At the last step, when i want to install OpenKai there is a error message:

nvidia@nvidia-desktop:~/Downloads/OpenKAI/build$ make all -j3 [ 2%] Building CXX object CMakeFiles/OpenKAI.dir/src/Arithmetic/PolyFit.cpp.o [ 2%] Building CXX object CMakeFiles/OpenKAI.dir/src/main.cpp.o [ 2%] Building CXX object CMakeFiles/OpenKAI.dir/src/Autopilot/ActionBase.cpp.o In file included from /home/nvidia/Downloads/OpenKAI/src/Startup/../Config/Module.h:49:0, from /home/nvidia/Downloads/OpenKAI/src/Startup/Startup.h:12, from /home/nvidia/Downloads/OpenKAI/src/main.h:11, from /home/nvidia/Downloads/OpenKAI/src/main.cpp:1: /home/nvidia/Downloads/OpenKAI/src/Startup/../Config/../SLAM/_ORB_SLAM2.h:16:10: fatal **error: System.h: No such file or directory #include <System.h>** ^~~~~~~~~~ compilation terminated. CMakeFiles/OpenKAI.dir/build.make:62: recipe for target 'CMakeFiles/OpenKAI.dir/src/main.cpp.o' failed make[2]: *** [CMakeFiles/OpenKAI.dir/src/main.cpp.o] Error 1 make[2]: *** Waiting for unfinished jobs.... /home/nvidia/Downloads/OpenKAI/src/Arithmetic/PolyFit.cpp:99:1: fatal error: opening dependency file CMakeFiles/OpenKAI.dir/src/Arithmetic/PolyFit.cpp.d: Permission denied } ^ compilation terminated. CMakeFiles/OpenKAI.dir/build.make:134: recipe for target 'CMakeFiles/OpenKAI.dir/src/Arithmetic/PolyFit.cpp.o' failed make[2]: *** [CMakeFiles/OpenKAI.dir/src/Arithmetic/PolyFit.cpp.o] Error 1 /home/nvidia/Downloads/OpenKAI/src/Autopilot/ActionBase.cpp:96:1: fatal error: opening dependency file CMakeFiles/OpenKAI.dir/src/Autopilot/ActionBase.cpp.d: Permission denied } ^ compilation terminated. CMakeFiles/OpenKAI.dir/build.make:158: recipe for target 'CMakeFiles/OpenKAI.dir/src/Autopilot/ActionBase.cpp.o' failed make[2]: *** [CMakeFiles/OpenKAI.dir/src/Autopilot/ActionBase.cpp.o] Error 1 CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/OpenKAI.dir/all' failed make[1]: *** [CMakeFiles/OpenKAI.dir/all] Error 2 Makefile:83: recipe for target 'all' failed make: *** [all] Error 2

i'm a total beginner and not really sure how to solve this problem, i didn't find related problems as well... and also when i try to do ccmake ../ there is no ZED Camera mentioned as in other Tutorials related to the OpenKai settings.
Or maybe it's not even necessary to install all the programs that are mentioned in the listed tutorial?

i would be very thankful if somebody has a suggestion or could help me with this problem :)

Hello, I'm graphic designer and my main purpose is supporting open source projects and make my portfolio better. So, I can contribute on your project with making logo design. I am waiting your feedback! Have a nice day! :)

Best Regards,
Baran Pirincal
Visual Communication Designer

hi,

I saw pose get in detect() of _ORB_SLAM2, but no one uses this pose. Am I correct?

Thanks a lot!

Hi,

I'm trying to run openKAI on the Nvidia Jetson TX2 flashed with the jetpack 3.3.1. I have installed OpenCV 3.4.0 on the TX2.

I'm getting the following error when I run the make command :

/home/nvidia/OpenKAI/src/Detector/_DNNdetect.cpp: In constructor ‘kai::_DNNdetect::_DNNdetect()’:
/home/nvidia/OpenKAI/src/Detector/_DNNdetect.cpp:22:15: error: ‘DNN_BACKEND_OPENCV’ is not a member of ‘cv::dnn’
m_iBackend = cv::dnn::DNN_BACKEND_OPENCV;
^
/home/nvidia/OpenKAI/src/Detector/_OpenPose.cpp: In constructor ‘kai::_OpenPose::_OpenPose()’:
/home/nvidia/OpenKAI/src/Detector/_OpenPose.cpp:19:15: error: ‘DNN_BACKEND_OPENCV’ is not a member of ‘cv::dnn’
m_iBackend = dnn::DNN_BACKEND_OPENCV;
^
CMakeFiles/OpenKAI.dir/build.make:1286: recipe for target 'CMakeFiles/OpenKAI.dir/src/Detector/_OpenPose.cpp.o' failed
make[2]: *** [CMakeFiles/OpenKAI.dir/src/Detector/_OpenPose.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/OpenKAI.dir/build.make:1262: recipe for target 'CMakeFiles/OpenKAI.dir/src/Detector/_DNNdetect.cpp.o' failed
make[2]: *** [CMakeFiles/OpenKAI.dir/src/Detector/_DNNdetect.cpp.o] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/OpenKAI.dir/all' failed
make[1]: *** [CMakeFiles/OpenKAI.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

I am following the steps of the documentation : http://docs.openkai.xyz/tegrabuild.html#jetson-inference-batch.

Any help regarding the issue would be great.
Thanks you!

If they already have one it should be present in each sample .kiss

Build on Ubuntu desktop systems
###Build on NVIDIA Jetson TX1

I want to open up multiple windows for visualization purpose in the same module. What would be the optimal way of approaching it?

I'm still new at all this and I need some help for this issue. I'm having problems flashing the apsync-tx1-openkai image from firmware.ardupilot.org into the TX1. While setting up the TX1 as companion computer I followed the instructions in ardupilot and I successfully downloaded JetPack 2.3.1 and flashed the official TX1 image. However when I tried to flash the apsync-tx1-openkai image it kept showing "USB communication failed. Check if device is in recovery", I typed "lsusb" in the terminal and it shows "ID 0955:7721 NVidia Corp." so I know that the device is in recovery mode. Has anyone encountered this problem as well?

Look https://github.com/PX4/Devguide/blob/master/en/ros/external_position_estimation.md

Dear @yankailab,

I had brief look into the code of OpenKai. I have noticed, that it maybe used with RealSense200 camera installed on Intel AeroBoard (which included FCU with ArduPilot). However I noticed, that it is using older librealsense.Correct? It means I would not be able to use it with newer librealsense for D400 series. Or you would rather recommend to utilize ZED Camera + Nvidia Jetson + PX4FCU with Ardupilot setup? Thank you for your recommendations. I know it is not an issue, but I did not find other place to ask. Thank you.

I have some questions to ask.

Thank you!

1.Is the APcopter_visualLanding class use detectNet for landing target? I find the mavlink landingTarget msg here.

2.I want to use AprilTag for automatic visual guided landing with pix copter,should I include this module to the APcopter_visualLanding class?I tried to add AprilTag_Landing to apLanding and add _AprilTags to the apCopter.kiss,but it doesn't work.Any help?Thanks.

apCopter.kiss file:

{
"name":"AprilTag_Landing",
"class":"_AprilTags",
"bInst":1,
"Window":"apView",
"FPS":30,
"_Stream":"ZED",
"_Automaton":"flight",
"family":"Tag16h5",
"err":0.1,
"lifeTime":300000,
"distThr":100,
"detInterval":500000,
"sizeLim":700,
"scaling":0.5,
}
{
"name":"apPilot",
"class":"_AutoPilot",
"_Automaton":"apState",
"bInst":1,
"FPS":20,
"Window":"apView",

{
	"name":"action",
	
	{
		"name":"apBase",
		"class":"APcopter_base",
		"bInst":1,
		"_Automaton":"apState",
		"Window":"apView",
		"_Mavlink":"apMavlink",
		"freqAtti":0,
		"freqGlobalPos":0,
	}
	{
		"name":"apLanding",
		"class":"APcopter_visualLanding",
		"bInst":1,
		"Window":"apView",
		"APcopter_base":"apBase",
		"_Stream":"ZED",
		"_AIbase":"detectNet",
		"_Automaton":"apState",
		"orientationX":1,
		"orientationY":1,
		"timeout":1000,
		"minSize":0.0,
		"maxSize":0.5,
	}	
}

}

Here is what I have so far:

1. Removed Opencv4tegra
2. Installed Opencv 3.1 on TX1 (Had problem installing Caffe with 3.2, hence 3.1!)
3. Got the image grabber working with opencv (python). Here is the VideoCapture input:

cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)192, height=(int)1080,format-(string)I420,framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")

4. Installed SSD

Now, how do I input the device ID to ssd_pascal_webcam.py?

Any leads will be appreciated. Thanks in advance!

This failure diagnostic is near-enough to opaque. Tell the user what went wrong :-)

apsync@apsync:~/GitHub/OpenKAI/OpenKAI/build$ ./OpenKAI ~/GitHub/OpenKAI/OpenKAI/kiss/apCopter.kiss
I1103 05:03:44.316718 21043 main.cpp:11] Kiss file:/home/apsync/GitHub/OpenKAI/OpenKAI/kiss/apCopter.kiss
I1103 05:03:44.318506 21043 Startup.cpp:60] OpenKAI: pApp->v("winMouse", &m_winMouse)
E1103 05:03:44.318590 21043 Startup.cpp:162] OpenKAI: Create instance failed: apView
E1103 05:03:44.319012 21043 Startup.cpp:162] OpenKAI: Create instance failed: Obstacle
I1103 05:03:44.319078 21043 _ThreadBase.cpp:43] apState: pK->v("FPS", &FPS)
I1103 05:03:44.319129 21043 APcopter_base.cpp:45] apBase: pCC->v("P", &cPID.m_P)
I1103 05:03:44.319165 21043 APcopter_base.cpp:46] apBase: pCC->v("I", &cPID.m_I)
I1103 05:03:44.319197 21043 APcopter_base.cpp:47] apBase: pCC->v("Imax", &cPID.m_Imax)
I1103 05:03:44.319227 21043 APcopter_base.cpp:48] apBase: pCC->v("D", &cPID.m_D)
I1103 05:03:44.319257 21043 APcopter_base.cpp:49] apBase: pCC->v("dT", &cPID.m_dT)
I1103 05:03:44.319283 21043 APcopter_base.cpp:50] apBase: pCC->v("rate", &cPID.m_rate)
I1103 05:03:44.319314 21043 APcopter_base.cpp:51] apBase: pCC->v("lim", &cPID.m_lim)
I1103 05:03:44.319339 21043 APcopter_base.cpp:57] apBase: pCC->v("P", &cPID.m_P)
I1103 05:03:44.319368 21043 APcopter_base.cpp:58] apBase: pCC->v("I", &cPID.m_I)
I1103 05:03:44.319398 21043 APcopter_base.cpp:59] apBase: pCC->v("Imax", &cPID.m_Imax)
I1103 05:03:44.319424 21043 APcopter_base.cpp:60] apBase: pCC->v("D", &cPID.m_D)
I1103 05:03:44.319459 21043 APcopter_base.cpp:61] apBase: pCC->v("dT", &cPID.m_dT)
I1103 05:03:44.319488 21043 APcopter_base.cpp:62] apBase: pCC->v("rate", &cPID.m_rate)
I1103 05:03:44.319517 21043 APcopter_base.cpp:63] apBase: pCC->v("lim", &cPID.m_lim)
I1103 05:03:44.319543 21043 APcopter_base.cpp:69] apBase: pCC->v("P", &cPID.m_P)
I1103 05:03:44.319567 21043 APcopter_base.cpp:70] apBase: pCC->v("I", &cPID.m_I)
I1103 05:03:44.319593 21043 APcopter_base.cpp:71] apBase: pCC->v("Imax", &cPID.m_Imax)
I1103 05:03:44.319618 21043 APcopter_base.cpp:72] apBase: pCC->v("D", &cPID.m_D)
I1103 05:03:44.319648 21043 APcopter_base.cpp:73] apBase: pCC->v("dT", &cPID.m_dT)
I1103 05:03:44.319674 21043 APcopter_base.cpp:74] apBase: pCC->v("rate", &cPID.m_rate)
I1103 05:03:44.319700 21043 APcopter_base.cpp:75] apBase: pCC->v("lim", &cPID.m_lim)
I1103 05:03:44.319725 21043 APcopter_base.cpp:80] apBase: pCC->v("P", &cPID.m_P)
I1103 05:03:44.319754 21043 APcopter_base.cpp:81] apBase: pCC->v("I", &cPID.m_I)
I1103 05:03:44.319780 21043 APcopter_base.cpp:82] apBase: pCC->v("Imax", &cPID.m_Imax)
I1103 05:03:44.319809 21043 APcopter_base.cpp:83] apBase: pCC->v("D", &cPID.m_D)
I1103 05:03:44.319835 21043 APcopter_base.cpp:84] apBase: pCC->v("dT", &cPID.m_dT)
I1103 05:03:44.319860 21043 APcopter_base.cpp:85] apBase: pCC->v("rate", &cPID.m_rate)
I1103 05:03:44.319890 21043 APcopter_base.cpp:86] apBase: pCC->v("lim", &cPID.m_lim)
E1103 05:03:44.320497 21043 _AutoPilot.cpp:93] apPilot: pA->link()
F1103 05:03:44.320544 21043 Startup.cpp:69] OpenKAI: m_pInst[i]->link()
*** Check failure stack trace: ***
    @       0x7f7aa9b718  google::LogMessage::Fail()
    @       0x7f7aa9d614  google::LogMessage::SendToLog()
    @       0x7f7aa9b290  google::LogMessage::Flush()
    @       0x7f7aa9deb4  google::LogMessageFatal::~LogMessageFatal()
    @           0x455c00  kai::Startup::start()
    @           0x423c34  main
    @       0x7f733a18a0  __libc_start_main
Aborted
apsync@apsync:~/GitHub/OpenKAI/OpenKAI/build$ ls

Specifying a configuration file which doesn't exist results in a silent failure:

apsync@apsync:~/GitHub/OpenKAI/OpenKAI/build$ ./OpenKAI ~/GitHub/OpenKAI/OpenKAI/kiss/apCopter.kis
I1103 05:02:05.968313 21038 main.cpp:11] Kiss file:/home/apsync/GitHub/OpenKAI/OpenKAI/kiss/apCopter.kis
apsync@apsync:~/GitHub/OpenKAI/OpenKAI/build$ 

Hi. @yankailab. Thanks for your great work! I am running this code on my jetson tx1. But I have no idea how to start. Could you please add a detailed README file for instructions. thanks!

Under build and run session both links seem to be broken.

###Build on NVIDIA Jetson TX1, ###Build on Ubuntu desktop systems

我看了以前的历史版本，这个系统以前是支持双目摄像头的-cudastero模块！但是后来可能考虑到性能和运算量问题删掉了！
目前zed cam 和inter realsensor 成本比较高，而且摄像头间距很小，深度探测距离很近，大约只有20m或更低。
我们自己设计了间距为40cm到80cm的大间距 同步usb3.0 uvc摄像头，以满足50到100米深度探测需求，主要用于固定翼飞机和无人车自动驾驶。这样基本可以替代激光雷达。
并且做了便携式计算机 i5+gtx960m，或者i5+gtx1060,满足运算需求。
希望这个系统加入对这种摄像头的支持，完成orb slam计算和双目立体深度计算，实现ardupilot 定位 定姿态 避障和路径规划功能

The output lines are way too long, and difficult to parse as they fly by.

Use 0-padding and length-specification in format strings to make output more consistent and shorten text so lines are a reasonable length:

e.g.
VISION_POSITION_DELTA dt=%u vx=%03.05f vy=%03.05f vz=%03.05f r=%03.05f p=%03.05f y=%03.05f cnf=%u

extern "C" {
#include "api.h"
}

在Darknet里面也没有api这个目录和文件，我是git clone https://github.com/pjreddie/darknet.git自己编译darknet。

if(USE_DARKNET)
add_definitions(-DUSE_DARKNET)
set(Darknet_root "/home/yuzq/darknet" CACHE STRING "")
set(OpenKAI_libDir ${OpenKAI_libDir} ${Darknet_root})
set(OpenKAI_include ${OpenKAI_include} "${Darknet_root}/api")
set(OpenKAI_lib ${OpenKAI_lib} darknet)
endif()

目前支持激光3d雷达slam吗？

Add TX2 (JetPack 3.0) compatibility to the project

Hey ,
I try to run OpenKai on my Nvidia TX2 with ZED ,but the web site ###Build on NVIDIA Jetson TX1 lost.Can you update the file and website?
Thx!!!

hi,

Why mavlink is commented out in inAPcopter_DNNavoid::update? which class implement the avoid function, DDNAvoid or SensorAvoid?

Thanks

Can I know that about drone hardware spec? except jetson TX1.