SRIP IITGN 2022 - Machine Learning - Prof. Nipun Batra

Jax is a Python library designed for high-performance ML research. Jax is nothing more than a numerical computing library, just like Numpy, but with some key improvements. It was developed by Google and used internally both by Google and Deepmind teams. References

• JAX Quickstart
• Youtube Playlist

It is a special case, multi-variate distribution. It's parameters are the mean vector which will have 2 elements and a covariance matrix. Check the implementation here

References

• Blog - Gaussian Processes
• Blog - 3D AND CONTOUR PLOTS OF THE BIVARIATE NORMAL DISTRIBUTION
• Video - Draw samples from a multivariate normal using numpy and scipy
• Blog - A Tutorial on Generating & Plotting 3D Gaussian Distributions with (Python/Numpy/Tensorflow/Pytorch) & (Matplotlib/Plotly)

The multivariate normal distribution is a multidimensional generalisation of the one-dimensional normal distribution . It represents the distribution of a multivariate random variable that is made up of multiple random variables that can be correlated with each other.

References

• Eric's Notes - ESTIMATING A MULTIVARIATE GAUSSIAN'S PARAMETERS BY GRADIENT DESCENT
• Blog - Sampling from a Multivariate Normal Distribution

We will create a neural network with one input layer, two hidden layer, and one output layer. We have a neural network with 784 inputs, the first hidden layer has 512 nodes while the second hidden layer consist of 256 nodes. The output layer has 10 node since we are solving a mnist classification problem, where there can be only 10 possible outputs. Check the implementation here

References

• Blog - Build Neural Network from Scratch
• Blog - Creating a Neural Network from Scratch in Python
• Article - Neural Networks
• Documentation - NN with JAX

The Bayesian approach describes probability as a measurement of belief in an event, using the prior and acquired knowledge from observed data. The goal is to update the probability distributions of the parameters by incorporating information about the parameters from observing the data.

References

• Blog - Bayesian Linear Regression
• Blog - Implementing Bayesian Linear Regression
• Video - Machine Learning: Bayesian Linear Regression
• BlackJAX - Bayesian Linear Regression