Abstract

Implementation of a Neural Network from scratch with a customized pipeline to parameter tuning and cross validation.

The algorithm was tested in a binary problem (Breast Cancer Dataset from sklearn) and a multiclass problem (mental imagery dataset - available at https://www.bbci.de/competition/iii/desc_V.html)

The algorithm used is a single hidden layer neural network, using sigmoid activation in all neurons. The bisection algorithm was used to find the best learning rate and speed up convergence.

The pipeline used to find the best configuration considering accuracy is described in the image below.

Remarks:

• The only parameter tested was the number of hidden layers
• The dataset was initally divided into training and test set (80% train and 20% test). The training set was then used for cross valitadion with k=5 folds. The best parameters selected in this phase were finally used to retrain the neural network in the entire training dataset
• The process described was repeted 5 times (results confidence)
• In the multiclass problem it was used Prinicipal Component Analysis with 30 components
• The data was normalized using z-score