Contains many functions useful in Machine Learning with tested scripts
-
- Ridge Regression: Least Squares + L_2 Regularization. * Closed Form solution using Normal Equation
- Lasso Regression * Implemented using Projected Gradient Descent
- Fourier Basis functions
- In the first example we have only one feature, thus we want to learn it's function. We use Fourier Basis and L1 Loss with L2 regularization since the data looks skewed.
2. The second example is a real dataset. The task is to predict the total number of violent crimes per 100K population from a set of (99) features. Polynomial basis was used capturing all sorts of properties of the cities and their population in non-linear fashion.
- Polynomial basis
- Validation data sorted out
- Data Normalization
-
Logistic regression with L1 Regularization
Soft margin SVM is regularized empirical risk minimization with hinge loss and L2 regularization.
Coordinate ascent method for solving the dual of the soft-margin SVM problem. For simplicity, we restrict to the case where the offset b is fixed at zero. The dual problem in this case is given by:
-
Handwritten digit classification using multi-class SVM with a Gaussian kernel. In order to solve the optimization problem for the SVM, we are using the MATLAB interface to the LIBSVM package (http://www.csie.ntu.edu.tw/~cjlin/libsvm/). Tested with libsvm 3.14
One vs All and One vs One techniques are compared. Check the respective .m files of the same
-
Exhaustive search for the best feature subset and permutation test.
-
Inspecting the datasets and the predictions made by the two methods. Comparing with a random classifier and then permutation test to check if the methods are better than the random classifier or not.
permutation test for testing whether p(x|y = 1) is equal to p(x|y = โ1) using the Fisher-score as the test-statistic (1000 permutations drawn uniform at random) and return the p-value.
-
Implements Gentle Boost using a real-valued decision stump as the weak base classifier.
-
Lloyd's algorithm for k-means clustering
Spectral clustering using unnormalized graph Laplacian
-
PCA vs Kernel PCA with a gaussian kernel
Zeeshan Khan Suri, Jyotsna Singh, Harshita Jhavar
This library started as successfully finished assignments from Machine Learning course from Saarland University. It also includes finished assignments form te Machine Learning course from Andrew Ng.
Licensing of the course assignments apply