1. Prajval Kumar Murali ([email protected])
2. Nicolò Bastianell ([email protected])
3. Luca Macchiusi ([email protected])

Detecting and classifying touch patterns with MiRo. Doing deeplearning using TensorFlow (keras).

1. Caress body top-bottom
2. Caress body bottom-top
3. Pat body
4. Fixed body
5. Pat Head
6. Fixed Head

1. Go straight forward
2. Go straight backward
3. Go in circle
4. Stop motion
5. Move head left to right
6. Stop head and reorient to center

For all gesture recognitions we have very good accuracy (94%) and for some high precision and for some low precision.

1. Pat Body (high false positive rate: we predict pat-body when it is actually not pat-body)

1. Pat Head
2. Fixed Head

Offline part: 2 files (collecting and storing data, training the RNN) Online part: 2 ROS-nodes (data input and machinelearning, miro actions)

Improve the precision and recall for all the gestures

Optimize the number of Hidden Neurons

After classification(recognizing) of gesture, we must have a voting system. As there is a sliding window, there are jumps in classified outputs, hence we must take the average (do the voting system)

Analyse the temporal performace (latency), ie, the time taken for action to happen after doing the gesture.

To configure MiRo with your workstation follow the setup guide on official website or https://github.com/EmaroLab/MIRO

#Collecting data after connecting to MiRo ROS node
python miro_touch_sub.py robot=rob01 name_file="name_of_gesture"
#Training the network
source ~/tensorflow/bin/activate
python DataSet.py #remember to change the path to dataset
python testset.py #to evaluate your model

#Data input and miro model node
source ~/tensorflow/bin/activate
python Data_input.py #remember to change the path to model :my_model_new.h5
#miro actions 
python miro_action.py #to move miro according to the gestures

Can be found here: https://www.youtube.com/watch?v=N7AkDnkxILg