Coursera Machine Learning Neural Network Back Propagation implementation in Scala with Scala NLP Breeze. Optimal value search fmincg is implemented with Breeze.optimize in Optimizer.scala.

Some MATLAB conventions may remain as it is used at Coursera, such as index is from 1, NOT 0. For the labels to classify digit, digit 1 is mapped to index 1, digit 0 is mapped to index 10. In Scala implementation, digit 1 is mapped to index 0, and digit 0 is mapped to index 9.

Represent how well/poor the network(theta1, theta2) performs with a cost function, which tells how far away the output of the network is from the correct digit. Feed back the cost to adjust (theta1, theta2) to bring down the cost by running gradient descent. To be able to reach the optima by the descent, the cost function is a convex function.

Estimate the effectiveness of the theta, calculate the penalty/error, or cost.

To feed the cost back and adjust the theta2 to bring the cost down, calculate the gradient of theta2. The optimization function from Breeze utilizes the gradient to run the gradient descent.

To feed the cost back and adjust the theta1 to bring the cost down, calculate the gradient of theta1. The optimization function from Breeze utilizes the gradient to run the gradient descent.

The original data contains 5000 digit images. Used 4900 data and 100 iterations of gradients for training the network, and used the remaining 100 for validations. Use the Breeze.optimize package DiffFuction to run the gradient descent(?).

Fed the validation data Z.CSV to test as in the Prediction results at YouTube.