Autonomous UAV Navigation Using Sample-based and Reinforcement Learning Methods
RRT star and RRT with Minimum snap
The RRT and RRT star method with Minimum snap trajectory generation. To see technical details, please take a look at our report.
Installation
- Create a new Python virtual environment and activate it.
- Run the following:
cd sample-based
cd src/
python setup.py install
To ensure the simulator is running, at least it should print the following:
Simulate.
Failure: Your quadrotor is out of control; it flew away with a position error greater than 20 meters.
indicating that the simulator works, no matter whether the simulation is successful or not.
After simulation, it should pop up in multiple Matplotlib windows. One of them is "Animation", and another one is "3D Path" which draws the entire path of the quadrotor in 3D space.
Running
Run the algorithm python core/code/sandbox.py
. By default, it should produce
----- Solving time optimization problem...
----- Solve maximum time reached.
Reach time before optimization:
[2. 4. 2.]
Reach time after optimization:
[1.63984032 4.43712496 1.92303472]
Simulate with time optimized.
Success: End reached.
Configurations
A list of configurable Python arguments can be seen when typing
python sandbox.py --help
which should include at least the following:
usage: sandbox.py [-h] [-s] [-e] [-f WORLD_FILE]
Accepts various command line arguments.
optional arguments:
-h, --help show this help message and exit
-s, --save_to_disk Save the animation to disk at ./data/out.mp4.
-e, --empty_world Create an empty world instead of from a pre-configured file.
-f WORLD_FILE, --world_file WORLD_FILE
World config JSON file location
You can alter the initial waypoint locations to test different shapes of trajectories in sandbox.py
:
# You will complete the implementation of the WaypointTraj object. It should
# work for any list of 3D coordinates, such as this example:
points = np.array([
[0, 0, 0],
[2.5, 0, 0],
[2.5, 5, 0],
[0, 5, 0]])
my_traj_noop = waypoint_traj.WaypointTraj(points, total_T=8, optimize_time=False)
my_traj = waypoint_traj.WaypointTraj(points, total_T=8, optimize_time=True)
You can also define different worlds with different obstacle locations by creating a new json file in src/core/code/worlds/
. The boundary of the world should be specified in
"bounds": {"extents": [0, 4.5, 0, 6.5, -1.0, 3.0]},
with the data format [xmin, xmax, ymin, ymax, zmin, zmax]
, and the obstacles
"blocks": [
{"extents": [0.0, 0.5, 0.0, 0.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [0.0, 0.5, 2.0, 2.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [0.0, 0.5, 4.0, 4.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [0.0, 0.5, 6.0, 6.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [2.0, 2.5, 0.0, 0.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [2.0, 2.5, 2.0, 2.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [2.0, 2.5, 4.0, 4.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [2.0, 2.5, 6.0, 6.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [4.0, 4.5, 0.0, 0.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [4.0, 4.5, 2.0, 2.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [4.0, 4.5, 4.0, 4.5, -1.0, 3.0], "color": [1, 0, 0]},
{"extents": [4.0, 4.5, 6.0, 6.5, -1.0, 3.0], "color": [1, 0, 0]}
]
with the data entry using the same format.
Results
You can get waypoints in src/core/code/output/rrt_3d.html
after you run sandbox.py
.
If you save the animation, you can get it in src/core/code/data/out.mp4
Here are example waypoints and trajectories of RRT star:
References
D. Mellinger and V. Kumar, "Minimum snap trajectory generation and control for quadrotors," 2011 IEEE International Conference on Robotics and Automation, 2011, pp. 2520-2525, doi: 10.1109/ICRA.2011.5980409. [IEEE Xplore]
Karaman, Sertac, and Emilio Frazzoli. "Sampling-based algorithms for optimal motion planning." The international journal of robotics research 30.7 (2011): 846-894.
LaValle, Steven M. "Rapidly-exploring random trees: A new tool for path planning." (1998): 98-11.
Q-learning for UAV Autonomous Navigation
This repository contains the source code for implementing Q-learning algorithm for ARDrone autonomous navigation.
Dependencies
- Ubuntu 16.04
- ROS Kinetic
- Gazebo 7
- ardrone_autonomy
- gym
- gym-gazebo
- Python 2.7
Installation
- Install all the dependencies
- Create a workspace for the motion planner
cd ~/catkin_ws/
catkin init
catkin_build
- Put the package in
~/catkin_ws/src
. Then in thesrc
directory, do
git clone https://github.com/pulver22/QLAB
git clone https://github.com/tum-vision/tum_ardrone
- Build the workspace
cd ..
catkin_make
Getting Started
To start the Q-learning training,
roslaunch uav_nav world.launch
roslaunch uav_nav start_qlearning.launch
Results
Reference
- Pham, Huy X., et al. Autonomous UAV Navigation using Reinforcement Learning
Contribution
- Xuanbiao Zhu: Implement RRT and RRT*
- Mengti Sun: Q-learning, PID Control, Gazebo Simulation
- Li Shen: LQR and Minimum Snap