• Ubuntu LTS 18.04

• ROS Melodic

• Setup PX4, QGC and MavROS

• JupyterLab

• Anaconda3

• Python 3.7+

• TensorFlow-gpu 2.5.0

• OpenCV-python 4.1.2.30

• Numpy, scipy, wget, tqdm, seaborn, Pillow, pandas, awscli, urllib3, mss

• NVIDIA Drivers

• CUDA 11.3

• CuDNN

This is a ROS package of YOLO with Tensorflow 2.x with DeepSORT for object detection and object tracking.

It contains ROS nodes for object detection and object detection with tracking.

The current repository is for a drone tracking another drone on PX4 but you should be able to adapt it for your own use case.

1. Guide for NVIDIA Drivers and CuDNN installation

2. Guide for CUDA installation

1. Anaconda3 Installation

2. JupyterLab Installation

3. Conda environment guide

Clone the repository then change to the TensorFlow_Yolo directory in terminal and use pip install -r requirements.txt

cd ~/catkin_ws/src
git clone https://github.com/dylantzx/yolo_ros.git --recursive
cd yolo_ros/src/TensorFlow_Yolo
conda activate <your_env>
pip install -r requirements.txt
catkin build yolo_ros
source ~/catkin_ws/devel/setup.bash

You can also install them individually with pip install <requirement> in your virtual environment

FPS.py - Contains a simple FPS class for FPS calculation

ImageConverter.py - Contains ImageConverter class that converts images received via GazeboROS Plugin of sensor_msgs type to a usable type for object detection.

ObjectTracker.py - Contains class to utilize DeepSORT for object tracking.

yolo_ros_detect_node.py - Main script that runs YOLO and TensorFlow 2.x with ROS for object detection

yolo_ros_track_node.py - Main script that runs YOLO and TensorFlow 2.x with ROS for object detection with tracking.

1. Go into your virtual environment

   conda activate <your_env>

2. To run object detection, run the launch file

   roslaunch yolo_ros yolo_detect.launch

3. To run object detection with tracking, run the launch file

   roslaunch yolo_ros yolo_track.launch

To train the YOLO model on your own custom dataset, you can refer to pythonlessons/TensorFlow-2.x-YOLOv3.

Place your images for training, validation and testing under TensorFlow_Yolo/IMAGES/

After getting your checkpoint files from training, place them under the TensorFlow_Yolo/checkpoints directory.

1. Go to TensorFlow_Yolo/yolov3/configs.py.
2. Ensure that the configs are suited to your use case. The important ones are:

   YOLO_TYPE (line 13)
   YOLO_CUSTOM_WEIGHTS (line 20)
   TRAIN_CLASSES (line 40)
   TRAIN_ANNOT_PATH (line 41)
   TRAIN_MODEL_NAME (line 44)
   TEST_ANNOT_PATH (line 57)
   

3. Go into the script that you will be using, for example yolo_ros_detect.py and change lines 32 -34 if required
4. Go to the launch file that you will be using
5. Remap the image_topic to your own ROStopic that is publishing the images
6. Go into TensorFlow_Yolo/yolov3/utils.py and edit sys.path.append('</abs/path/to/TensorFlow_Yolo>) with your own absolute path to the repository.
7. In TensorFlow_Yolo/yolov3/utils.py, also edit checkpoint_path with your own absolute path to the checkpoint folder.

The evaluation script is found in the TensorFlow_Yolo/evaluation/main.py.

Change the path and names on lines 34 - 36 into your own.

In your terminal, change directory into TensorFlow_Yolo/evaluation and run python main.py

Upon successful execution, you should get an excel spreadsheet under TensorFlow_Yolo/exports/ with your evaluation results.