This repository contains all the necessary material to train a PiCar-X to perform the task of "cleaning" a squared environment from cubes. The training of the agent is performed using Deep Reinforcement Learning on a simulated CoppeliaSim environment.

Jonathan Collu*, Riccardo Majellaro*, Irina Mona Epure, Diego Barreiro and Ayodele Adetunji

To run these scripts, a Python 3 environment is required, together with the necessary packages specified in the requirements.txt file. In order to install the requirements, run the following command from the main directory:

pip install -r requirements.txt

Run the following command from the main directory

./upload_file.sh <IP address of the picar> <local filepath>

For the training on the simulation it is necessary to copy the following files in the CoppeliaSim src directory: Model.py, Reinforce.py, agent.py, color_detection.py and env.py. The files start.py and run.py must be copied in the CoppeliaSim main folder. Run the command below from the main directory

python run.py -run_name <run_name> -cp_name <checkpoint_name> -epochs <epochs_num> -M <traces_per_epoch> -T <trace_len> -gamma <discount>

An example is shown in train.sh.

Run the command below from the main directory

python evaluate.py -w <your_weights.pt>

From the main directory run the command:

python server.py -parameters <your_weights.pt>`

then connect to the robot via ssh and run on it the following command:

python client.py -host_address <server_IP_address>`