Drone pythone

DroNet: Learning to fly by driving

This repository contains the code used to train and evaluate DroNet, a convolutional neural network that can safely drive a drone along the streets of a city.

Citing

If you use DroNet in an academic context, please cite the following publication:

Paper: DroNet:Learning to Fly by Driving

Video: YouTube

@article{Loquercio_2018,
	doi = {10.1109/lra.2018.2795643},
	year = 2018,
	author = {Antonio Loquercio and Ana Isabel Maqueda and Carlos R. Del Blanco and Davide Scaramuzza},
	title = {Dronet: Learning to Fly by Driving},
	journal = {{IEEE} Robotics and Automation Letters}
}

Introduction

Due to the danger that flying a drone can cause in an urban environment, collecting training data results impossible. For that reason, DroNet learns how to fly by imitating the behavior of manned vehicles that are already integrated in such environment. It produces a steering angle and a collision probability for the current input image captured by a forward-looking camera. Then, these high-level commands are transferred to control commands so that the drone keeps navigating, while avoiding obstacles.

Model

DroNet has been designed as a forked CNN that predicts, from a single 200×200 frame in gray-scale, a steering angle and a collision probability. The shared part of the architecture consists of a fast ResNet-8 with 3 residual blocks, followed by dropout and ReLU non-linearity. After them, the network branches into two separated fully-connected layers, one to carry out steering prediction, and the other one to infer collision probability. See cnn_models.py for more details.

Data

In order to learn steering angles, the publicly available Udacity dataset has been used. It provides several hours of video recorded from a car. We additionally recorded an outdoor dataset to learn the probability of collision by riding a bicycle in the streets of our city.

Running the code

Software requirements

This code has been tested on Ubuntu 14.04, and on Python 3.4.

Dependencies:

TensorFlow 1.5.0
Keras 2.1.4 (Make sure that the Keras version is correct!)
NumPy 1.12.1
OpenCV 3.1.0
scikit-learn 0.18.1
Python gflags

Data preparation

Steering data

Once Udacity dataset is downloaded, extract the contents from each rosbag/experiment. To have instructions on how to do it, please follow the instructions from the udacity-driving-reader, to dump images + CSV. After extraction, you will only need center/ folder and interpolated.csv file from each experiment to create the steeering dataset. To prepare the data in the format required by our implementation, follow these steps:

Rename the center/ folder to images/.
Process the interpolated.csv file to extract data corresponding only to the central camera. It is important to sync images and the corresponding steerings matching their timestamps. You can use the script time_stamp_matching.py to accomplish this step.

The final structure of the steering dataset should look like this:

training/
    HMB_1_3900/*
        images/
        sync_steering.txt
    HMB_2/
    HMB_4/
    HMB_5/
    HMB_6/
validation/
    HMB_1_501/*
testing/
    HMB_3/

*Since Udacity dataset does not provide validation experiments, we split HMB_1 so that the first 3900 samples are used for training and the rest for validation.

Collision data

Collision dataset is ready to be used after downloading. It can be directly downloaded from here. Its structure is as follows:

training/
    DSCN2561/
        images/
        labels.txt
    DSCN2562/
    ...
    GOPR0387/
validation/
    DSCN2682/
    GOPR0227/
testing/
    DSCN2571/
    GOPR0200/
    ...
    GOPR0386/

Finally, merge the training/, validation/ and testing/ directories from both datasets, and use them to train and evalutate DroNet.

Training DroNet

Train DroNet from scratch:

python cnn.py [flags]

Use [flags] to set batch size, number of epochs, dataset directories, etc. Check common_flags.py to see the description of each flag, and the default values we used for our experiments.

Example:

python cnn.py --experiment_rootdir='./model/test_1' --train_dir='../training' --val_dir='../validation' --batch_size=16 --epochs=150 --log_rate=25

Fine-tune DroNet by loading a pre-trained model:

python cnn.py --restore_model=True --experiment_rootdir='./model' --weights_fname='model_weights.h5'

The pre-trained model must be in the directory you indicate in --experiment_rootdir.

Evaluating DroNet

We evaluate our model on the testing data from each dataset.

python evaluation.py [flags]

Example:

python evaluation.py --experiment_rootdir='./model' --weights_fname='model_weights.h5' --test_dir='../testing'

Download our trained model

A folder containing the trained model that we used in our real world experiments can be download here: best_model. To evaluate it, use the following command:

python evaluation.py --experiment_rootdir=PATH_TO_FOLDER --weights_fname=best_weights.h5 --test_dir='../testing'

Test dronet on a real robot

In the folder drone_control you can find instructions on how to use DroNet to fly a Bebop drone.

uzh-rpg / rpg_public_dronet Goto Github PK

rpg_public_dronet's Introduction

DroNet: Learning to fly by driving

Citing

Introduction

Model

Data

Running the code

Software requirements

Data preparation

Steering data

Collision data

Training DroNet

Evaluating DroNet

Download our trained model

Test dronet on a real robot

rpg_public_dronet's People

Stargazers

Watchers

Forkers

rpg_public_dronet's Issues

Build batch of image data

yoni@yoni-ThinkPad-E470:~/bebop_ws/dronet/dronet_control$ catkin build --this

Profile: default Extending: [cached] /opt/ros/kinetic Workspace: /home/yoni/bebop_ws

Source Space: [exists] /home/yoni/bebop_ws/src Log Space: [exists] /home/yoni/bebop_ws/logs Build Space: [exists] /home/yoni/bebop_ws/build Devel Space: [exists] /home/yoni/bebop_ws/devel Install Space: [unused] /home/yoni/bebop_ws/install DESTDIR: [unused] None

Devel Space Layout: linked Install Space Layout: None

Additional CMake Args: None Additional Make Args: None Additional catkin Make Args: None Internal Make Job Server: True Cache Job Environments: False

Whitelisted Packages: None Blacklisted Packages: None

Workspace configuration appears valid.

Model reconstruction from JSON file

Load weights into the new model

Load the image file

Predict

there is not python3.4 with keras2.1.4 and tensorflow1.0.1 ; so , i install python3.5 with keras2.1.4 and tensorflow1.0.1

there is not python3.4 with keras2.1.4 and tensorflow1.0.1 ; so , i install python3.5 with keras2.1.4 and tensorflow1.0.1 . Then, happen follow error:

Recommend Projects

Recommend Topics

Recommend Org

Profile: default
Extending: [cached] /opt/ros/kinetic
Workspace: /home/yoni/bebop_ws

Source Space: [exists] /home/yoni/bebop_ws/src
Log Space: [exists] /home/yoni/bebop_ws/logs
Build Space: [exists] /home/yoni/bebop_ws/build
Devel Space: [exists] /home/yoni/bebop_ws/devel
Install Space: [unused] /home/yoni/bebop_ws/install
DESTDIR: [unused] None

Devel Space Layout: linked
Install Space Layout: None

Additional CMake Args: None
Additional Make Args: None
Additional catkin Make Args: None
Internal Make Job Server: True
Cache Job Environments: False

Whitelisted Packages: None
Blacklisted Packages: None