Things learned :

• Data Selection: Consider what data is available, what data is missing and what data can be removed.
• Data Preprocessing: Organize selected data by formatting, cleaning and sampling from it.
• Data Transformation: Transform preprocessed data ready for machine learning by engineering features using scaling, attribute decomposition and attribute aggregation.
• Used numpy, matplotlib, numpy and sklearn libraries to handle, visualize and manipulate data

Things learned :

• What is SVM? Concepts and theory
• Implementation in python: SVM and Kernel SVM
• Different Kernel SVMs and comparison

Things learned : (tutorial helps from machinelearningmastery)

• How Decision Trees work
• Splits made on the basis of entropy or Gini Index
• Implementation in sklearn and from scratch

Things learned : (Fundamentals of Deep Learning, Chapter 3)

• Logistic Regression on MNIST Data
• Feed Forward Network on MNIST data and comparison
• Tensorflow implementation : using variable scope and name scope for network

Things learned

• Challenges with Gradient Descent: Local Minima and their effect in deep learning error surfaces
• Momentum based Optimization: keeping memory of grdients for smoother error surfaces
• Learning Rate Adaptation: (1) Adagrad (2) RMSProp (3) Adam
• Adagrad accumaltes and adapts the global learning rate using istorical gradients
• RMSProp is exponentially weighted moving average of gradients: it enables us to "toss out" measurements we made a long time ago
• Adam is the variant of both RMSProp and AutoGrad

Watched Documentaries and videos relating ML and DL

• Watched AlphaGo Documentary to learn how the game was built to be smart enough to beat the world champion
• Watched Jeremy Howard's account on machine learning and computer vision integrated with machine learning
• Fei-Fei Li: How we're teaching computers to understand pictures

Things learned

• Implementation of Naive Bayes Classifier in python
• Class probability and attribute probability based classifier
• Functions for class probabilities and attribute probabilities
• More work required on the concepts

Things learned

• How to think about a machine learning problem
• Howto think about the output and to analyse what we want to predict from the model
• While thinking about features and input values, think about the availability of all the data at deployment time

Things learned

• How to use speech recognition library in python for speech_recognition from microphone
• Develop a small guessing game based on the recognized speech from microphone
• Theory of how it all works

Things learned

• What are convolutional neural networks and what was the need?
• What are filters and feature maps and how convolution helps extracting features
• Implementation on MNIST data using Tensorflow

Things learned

• Batch normalization and how it is helpful for training
• How CIFAR 10 dataset is handled by using batch normalization
• Implementation of the network

Things learned

• Embedding and Representation Learning: A way to escape the curse of dimensionality
• Principal Component Analysis: concepts and mathematical formulation study
• AutoEncoders: Introduction and basic concepts

Things learned

• Denoising Autoencoders: More Robust Autoencoders
• Introducing Sparsity in Autoencoders
• When context is important in representations: English language as an example and their representation learning
• Coded an mnist autoencoder (using dense layers) in keras

Things learned

• Implementing Autoencoders using convolutional neural networks
• Using convo layers works better than fully connected layer in terms of reconstruction

Pytorch tutorial:

Things learned

• pytorch methods for building neural networks
• Data loading and manipulations to tensors in pytorch
• Autograd and backprop concepts in pytorch

Pytorch tutorial:

Things learned

• How to create your own custom dataloader
• How to transform your data to make it same
• How to use DataLoader pytorch class to enable batching, shuffling and parallel loading of data

Pytorch tutorial:

Things learned

• Main Features of Pytorch: n-dimensional tensors, kinda like numpy arrays but can run on GPU. Second important is Automatic differentiation for building and training networks
• Deeper understanding of Autograd and how pytorch builds computational graphs to computer gradients and weight updates
• Difference between Tensorflow and Pytorch: Dynamic and static graph
• Hw to build custom nn modules and optimizers

Pytorch tutorial:

Things learned

• What is transfer learning and the importance of transfer learning: fine-tuning, as a fixed feature extractor
• Constraints of Transfer learning
• Implementation of transfer learning using pytorch (help from pytorch tutorial)

Lecture

The course took 5 days with extensive lab introduction and practice