A ROS package to run Reinforcement Learning experiments, particularly pick and place tasks, on the TIAGo robot. Uses Gazebo, Rviz and MoveIt! (for motion planning)

Tested on Ubuntu 18.04 only. Beware! Instructions assume familiarity with the ROS packages system.

• Install ROS Melodic + TIAGo
  • http://wiki.ros.org/Robots/TIAGo/Tutorials/Installation/InstallUbuntuAndROS
• Install openai_ros package into your TIAGo workspace

   cd /home/user/tiago_public_ws/src
   git clone https://bitbucket.org/theconstructcore/openai_ros.git
   cd openai_ros;git checkout version2
   cd /home/user/tiago_public_ws;catkin_make;source devel/setup.bash

• Install tiago_gym package into your TIAGo workspace

   cd /home/user/tiago_public_ws/src
   git clone https://github.com/edluffy/tiago_gym
   cd /home/user/tiago_public_ws;catkin_make;source devel/setup.bash

• Launch an environment!
  • roslaunch tiago_gym start_gym.launch
  • Gazebo and Rviz should launch similarly to gif above.
  • To speed up simulation you can run the command gz physics –u 0 -s 0.0025 in a separate terminal.
• Running custom training scripts
  • Launch with roslaunch tiago_gym start_gym.launch train:=false
  • Run in a separate terminal python my_training_script.py. Example snippets are shown in the Environments section.

TiagoSimpleEnv-v0	TiagoReachEnv-v0

This is a simple test environment in which the robot gripper must move to a discrete goal position in 3D space (essentially a 3D gridworld). Example usage:

import tiago_simple_env
from agents import dqn

env = gym.make('TiagoSimpleEnv-v0')
o_dims = len(env.observation_space.sample())
a_dims = env.action_space.n

agent = dqn.DQN(env, input_size=o_dims,
        output_size=o_dims, alpha=0.01, epsilon_decay=0.95)
agent.run(100)

A continuous action environment – robot can move a vector distance in any direction to get to the goal. Example usage:

import tiago_reach_env
from agents import ddpg

env = gym.make('TiagoReachEnv-v0')
o_dims = len(env.observation_space.sample())
a_dims = env.action_space.shape[0]
a_high = env.action_space.high
    
agent = ddpg.DDPG(env, o_dims, a_dims, a_high)
agent.run(100)

• Tensorflow implementations of DQN and DDPG can be found in scripts/agents.