In this lab, you'll explore the classic MNIST dataset of handwritten digits. While not as large as the previous dataset on facial image recognition, it still provides a 64 dimensional dataset that is ripe for feature reduction.

You will be able to:

• Train a baseline classifier using sci-kit learn
• Use grid search to optimize the hyperparameters of a classifier
• Perform dimensionality reduction using PCA
• Calculate the time savings and performance gains of layering in PCA as a preprocessing step in machine learning pipelines

To start, load the dataset using sklearn.datasets.load_digits.

#Your code here

Now that the dataset is loaded, display the images of the first 20 pictures.

#Your code here

Now it's time to fit an initial baseline model to compare against. Fit a support vector machine to the dataset using sklearn.sv.SVC(). Be sure to perform a train test split, record the training time and print the training and testing accuracy of the model.

#Your code here

Refine the initial model slightly by using a grid search to tune the hyperparameters. The two most important parameters to adjust are "C" and "gamma". Once again, be sure to record the training time as well as the train and test accuracy.

#Your code here
# ⏰ Your code may take several minutes to run

Now that you've fit a baseline classifier, it's time to explore the impacts of using PCA as a preprocessing technique. To start, perform PCA on X_train. (Be sure to only fit PCA to X_train; you don't want to leak any information from the test set.) Also, don't reduce the number of features quite yet. You'll determine the number of features needed to account for 95% of the overall variance momentarily.

#Your code here

In order to determine the number of features you wish to reduce the dataset to, it is sensible to plot the overall variance accounted for by the first n principle components. Create a graph of the variance explained versus the number of principle components.

#Your code here

Great! Now determine the number of features needed to capture 95% of the dataset's overall variance.

#Your code here

Use your knowledge to reproject the dataset into a lower dimensional space using PCA.

#Your code here

Now, refit a classification model to the compressed dataset. Be sure to time the required training time, as well as the test and training accuracy.

#Your code here

Finally, use grid search to find optimal hyperparameters for the classifier on the reduced dataset. Be sure to record the time required to fit the model, the optimal hyperparameters and the test and train accuracy of the resulting model.

#Your code here
# ⏰ Your code may take several minutes to run

Well done! In this lab, you employed PCA to reduce a high dimensional dataset. With this, you observed the potential cost benefits required to train a model and performance gains of the model itself.