The code for the data summary is found in the second cell of the Jupyter notebook. I used the pandas library to calculate summary statistics for the data provided by Udacity:

• The training set featured 34799 examples.
• The validation set featured 4410 examples.
• The test set featured 4410 examples.
• Each traffic sign image was 32x32 pixels.
• There were 43 distinct labels for the data.

The code for the data visualization is found in the third cell of the Jupyter notebook. I created a bar chart illustrating the frequency of each sign in the training data.

Please refer to the labels lookup table below for y-axis tick values.

The code for this step is written in the fifth and sixth cells of the Jupyter notebook.

Preprocessing of the images included grayscaling and normalizing all images.

I grayscaled the images due to the variable color distribution of signs with the same label. For instance, a no entry sign (labeled 17 in our data) is known to the human eye to be red. However, there are other variables that will influence an image's color data (and thus influencing a convolutional neural network). Brightness and lumonisty are two examples of variables that may alter an image's color data and make it harder to train with a LeNet model architecture.

Normalization is done to scale our input values down to be within the same range as our weight matrices and learning rate. Consider a convolutional neural network that does not use normalized inputs. The learning rate may overcompensate or undercompensate corrections since the ranges of each feature are vastly different. This leads to oscillation and a more difficult time in finding the minimum loss.

Source: Why do we need to normalize the images before we put them into CNN?

An example of an image before and after preprocessing:

The LeNet architecture was used for image recognition. I chose this architecture for its effectiveness, its wide use and Udacity's help in learning the architecture. The code for this architecture is found in the seventh cell of the Jupyer notebook. The architecture consists of the following layers:

The code to train the model is found in the ninth cell of the Jupyter notebook.

I set the learning rate to 0.0011 found through trial and error. I used the cross-entropy cost function and the Adam Optimizer to minimize that cost.

Optimizing the training model was an iterative process full of mistakes, trial and error.

To begin, I set the number of epochs to 10, the standard deviation of the weight initializations to 0.1 and forgot to add dropout to my model. My first few validation accuracies rose but starkly dropped at the eighth epoch. I suspected overfitting.

After Googling ways to remedy overfitting, I remembered the dropout module at Udacity. I looked at existing code of mine where I used dropout and implemented it into my code. However, this added a new hyperparameter: the keep probabilty.

I began with a keep probability of 80% but quickly lowered that to 50% then brought it up a bit to 60%. I found 60% worked well but still left me around 85% validation accuracy.

After reading many Quora questions regarding CNN accuracy, I realized that weight initialization was important. I changed my sigma for weight initialization to 0.05 and increased the number of epochs to 30 (to give my CNN more time to find optimal parameters).

The change of sigma was exactly what the training model needed. My validation accuracy easily hit 93% and my test accuracy hit 92%.

Here are six German traffic signs that I found on the internet and edited (cropped and resized using Microsoft Paint):

Speed limits: Speed limits may be difficult to classify since they share the same attributes: circular, white outer border, colored inner border, white inside and numbers that end with zero inside. Given these similarities, the neural network may confuse one for the other.

Triangular signs: Similarly to speed limits, the triangular speed limits share shape, border colors and inner colors. They may also be confused with other triangular signs.

Keep right vs. Keep left: Given the similar shape and diagram of the keep right/left signs, these two may be confused for one another.

The code to run the additional tests is found 19th cell of the Jupyter notebook. The following predictions were made:

The model is only 66.67% accurate as compared to the 92.1% accuracy of the test set. This can be attributed to the small sample size of these additional tests. Additionally, the bumpy road and bicycles crossing signs were only trained with 570 images total. This may have led to a undertrained model for these specific images.

Predictions for the first image (Speed limit (120km/h)):

Predictions for the second image (Speed limit (30km/h)):

Predictions for the third image (Bumpy road):

Predictions for the fourth image (Keep left):

Predictions for fifth image (No entry):

Predictions for sixth image (Right-of-way at the next intersection):