Build a Traffic Sign Recognition Project

The goals / steps of this project are the following:

• Load the data set (see below for links to the project data set)
• Explore, summarize and visualize the data set
• Design, train and test a model architecture
• Use the model to make predictions on new images
• Analyze the softmax probabilities of the new images
• Summarize the results with a written report

You're reading it! and here is a link to my project code

I used python and the numpy library to calculate summary statistics of the traffic signs data set:

• The size of training set is 34,799
• The size of the validation set is 4,410
• The size of test set is 12,630
• The shape of a traffic sign image is (32, 32, 3)
• The number of unique classes/labels in the data set is 43

Here is an exploratory visualization of the data set. It is a bar chart showing the number of instances of each class in the training, validation and test sets.

I chose to leave the images in RGB color space because that increases the depth of the network and generally allows more data through to help classify an image.

To normalize the images, found in code cell #12 in the jupyter notebook, I subtracted the minimum image threshold from each image in the dataset, then divided the result by the range of brightness/color in the entire dataset. This helped to remove outliers from the dataset that could decrease the classifier's accuracy. After numerous tests, I found that less normalization provided higher accuracy, so I chose to leave the preprocessing to outlier reduction.

In addition to normalization, I generated additional data in the training set to account for classes that had fewer than 600 occurrences in the dataset. The data was augmented based on the following characteristics:

• Rotating the Image
• Skewing the Image
• Shearing the Image
• Changing Image Brightness

Each value was picked at random to ensure that no two generated images were the same. Here is an example of a speed limit sign image before augmentation:

And this is one example of augmenting the image to add to the training dataset:

By augmenting all classes in the training set, I was able to add 6,661 images to the dataset resulting in the following distribution among classes:

My final model consisted of the following layers:

To train the model, I used the following:

• Adam Optimizer
• Batch Size is 80
• Number of Epochs is 50
• Learning Rate is 0.001
• Dropout Keep Probability is 0.4

I chose the Adam Optimizer because it allowed for moving averages of the hyperparameters to result in momentum across epochs compared to the static hyperparameters of gradient descent. After lots of tuning, I settled on a batch size of 80 and a learning rate of 0.001 simply because this combination resulted in the highest accuracy. A keep probability of 0.4 for the dropout function worked best to prevent overfiting since my network uses 64 filters at the widest point. The balance of overfitting in the network architecture and dropout in training result in a balanced classifier.

A graph showing the training and validation accuracy across training epochs is shown below:

My final model results (calculated in code cell #20 of the jupyter notebook) were:

• training set accuracy of 1.000
• validation set accuracy of 0.973
• test set accuracy of 0.954

If an iterative approach was chosen:

• What was the first architecture that was tried and why was it chosen?

  I started this project with the LeNet-5 architecture. This was based on my experience with the LeNet CNN for handwriting recognition. I tested it on traffic signs and got ~80% accuracy so I decided to modify this network to increase accuracy.

• What were some problems with the initial architecture?

  The LeNet model was underfitting the data based on the lower accuracy.

• How was the architecture adjusted and why was it adjusted? Typical adjustments could include choosing a different model architecture, adding or taking away layers (pooling, dropout, convolution, etc), using an activation function or changing the activation function. One common justification for adjusting an architecture would be due to overfitting or underfitting. A high accuracy on the training set but low accuracy on the validation set indicates over fitting; a low accuracy on both sets indicates under fitting.

  To adjust the architecture, I first modified the pooling operations between the two 5x5 convolutions. The default for this was both max pooling, but I changed it to average pooling followed by max pooling in the next layer. After adding the pooling adjustments and the hyperparameter tuning discussed above, I was getting overfitting with high training accuracy but much lower validation accuracy. To combat this, I also added a dropout function in the fully connected layers to help compensate the now overfitted dataset.

• Which parameters were tuned? How were they adjusted and why?

  I tuned the number of epochs, the batch size, and the keep probability. Smaller batch sizes yielded higher accuracies so I chose to reduce the batch size from 128 to 80. I also increased the number of epochs from 10 to 50. This didn't provide much improvement in the long run, but helped to identify when the classifier reached it peak accuracy. Finally I started with a keep probability of 0.5, but gradually reduced it to 0.4 to improve validation accuracy.

• What are some of the important design choices and why were they chosen? For example, why might a convolution layer work well with this problem? How might a dropout layer help with creating a successful model?

  I found it was important to overfit the data within the convolutions, then apply dropout in the fully connected layers. Since this was a fairly large and diverse dataset, overfitting the data helped to identify as many characteristics of each class as possible. The dropout was then essential in the fully connected layer to randomize what information is retained and overall resulted in a very accurate classifier.

Here are five German traffic signs that I found on the web:

Overall, each image is very clear and should provide a good test case for the classifier. The difficulty comes in when the classifier needs to dissern between the similarities of each image.

For example, both of the images on the left are round with similar coloring, the only difference is the "30" in the top image. Each image also has a counterpart of the same color and shape, and it is an accurate test of the classifier because not only does the classifier need to recognize basic sign types, but this shows its ability to detect small differences in signs that result in completely different classifications.

Here are the results of the prediction:

The model was able to correctly guess 5 of the 6 traffic signs, which gives an accuracy of ~83%. This is lower than the accuracy on the test set, but I believe that is due to the classifier struggling with text recognition in the speed limit signs. I believe this can be solved by adding more training data to the speed limit signs as well as increasing their resolution so the classifier can extract more features from the text.

The code for making predictions on my final model is located in code cell #24 in the jupyter notebook.

For the first image, the model is relatively sure that this is a speed limit sign (probability of 0.852), and the image does contain a speed limit sign although it is 30km/h not the predicted 20km/h. The correct prediction was 2nd on the softmax probabilities (probability of 0.148). The top five soft max probabilities were:

For the second image, the model is completely sure that this is a bumpy road sign (probability of 1.000), and the image does contain a bumpy road sign. The top five soft max probabilities were:

For the third image, the model is completely sure that this is an ahead only sign (probability of 1.000), and the image does contain an ahead only sign. The top five soft max probabilities were:

For the fourth image, the model is completely sure that this is a no vehicles sign (probability of 1.000), and the image does contain a no vehicles sign. The top five soft max probabilities were:

For the fifth image, the model is completely sure that this is a go straight or left sign (probability of 1.000), and the image does contain a go straight or left sign. The top five soft max probabilities were:

For the sixth image, the model is completely sure that this is a general caution sign (probability of 1.000), and the image does contain a general caution sign. The top five soft max probabilities were: