Machine learning is how to get computers to automatically learn and improve with experience. Experience comes in the form of data, improvement is with respect to some performance metric, and learning is done by a learning algorithm. There are always computational constraints, such as the architecture, computation time, bandwidth limitations, and so on. So we can more precisely restate the goal thus: to construct learning algorithms that use data to improve with respect to a performance metric and do so under computational constraints.

We will focus on principles of statistical machine learning in the prediction problems, regression and classification. Conspicuously absent is any Bayesian methodology, hidden Markov models, unsupervised learning including density estimation, clustering, dimension reduction, network modelling, etc. This course is not a broad overview of all of machine learning, but rather a tour of the key ideas in machine learning as told through these prediction tasks. Typically, I have students tell me something along the lines of "I thought machine learning was about [insert random methodology here]". Machine learning is a field, like physical chemistry or creative literature. It is not defined by a couple of methods or a single task, and cannot be taught in a single quarter. With that said, I want this course to lay the foundation for a rich understanding of machine learning.

1. Class time will be for lectures and labs. These will be posted in advance in their sections. Labs are not graded, and you will have an opportunity to go over your results in class.
2. Homework will be due every week or so, starting with the second week. (50% of grade)
3. Discussions and questions should be posted on the slack site.
4. A final project will be due at the end of the class. Instructions can be found here. (50% of grade)
5. You will need to install Python and the necessary packages to participate in this class. See the following instructions for the installation.

• James: Wednesdays 9am-12pm in MSB 4107
• Chunzhe: Monday 1-3pm in MSB 1117
• Huang: Friday 1-3pm in MSB 1117

• Required: Elements of Statistical Learning by Hastie, Tibshirani, and Friedman
• Recommended: Python for Probability, Statistics, and Machine Learning by Unpingco
• Recommended: Convex Optimization by Boyd and Vandenberghe

• Homework 1: assignment and solutions
• Homework 2: assignment and solutions
• Homework 3: assignment

Principles: Over/under-fitting, training and testing, losses
Reading: ESL Chapter 2

Principles: Convex relaxation, computational intractability in subset selection
Reading: ESL Chapter 3, Boyd Chapter 1

Principles: Surrogate losses, generative and discriminative methods
Reading: ESL Chapter 4

Principles: Compression, Dimension Reduction Reading: ESL Chapter 14

Principles: Feature extraction, the kernel trick, analysis/sythesis duality
Reading: ESL Chapter 5

Principles: Interpretable models, statistical complexity, learning from experts
Reading: ESL Chapters 7, 8

Principles: multi-layer architectures, non-convex optimization, online algorithms

Principles: Markov network, dynamical models

├── LICENSE
├── Makefile           <- Makefile with commands like `make data` or `make train`
├── README.md          <- The top-level README for developers using this project.
├── data
│   ├── external       <- Data from third party sources.
│   ├── interim        <- Intermediate data that has been transformed.
│   ├── processed      <- The final, canonical data sets for modeling.
│   └── raw            <- The original, immutable data dump.
│
├── labs               <- notebook files for course labs (6 labs)
│
├── homeworks          <- notebook files for homeworks (6 hws)
│
├── lectures           <- notebook files for lectures (naming convention 
│
├── references         <- Reference material, pdfs, etc.
│
├── reports            <- Generated analysis as HTML, PDF, LaTeX, etc.
│   └── figures        <- Generated graphics and figures to be used in reporting
│
├── src                <- Source code for use in this project.
│   ├── __init__.py    <- Makes src a Python module
│   │
│   ├── data           <- Scripts to download or generate data
│   │   └── make_dataset.py
│   │
│   ├── features       <- Scripts to turn raw data into features for modeling
│   │   └── build_features.py
│   │
│   ├── models         <- Scripts to train models and then use trained models to make
│   │   │                 predictions
│   │   ├── predict_model.py
│   │   └── train_model.py
│   │
│   └── visualization  <- Scripts to create exploratory and results oriented visualizations
│       └── visualize.py
│
└── tox.ini            <- tox file with settings for running tox; see tox.testrun.org

Project based on the cookiecutter data science project template. #cookiecutterdatascience