Instructor: Justin Sirignano
Teaching Assistants: Logan Courtney, Raj Kataria, Xiaobo Dong

What is Deep Learning?

Deep learning has revolutionized image recognition, speech recognition, and natural language processing. There's also growing interest in applying deep learning to science, engineering, medicine, and finance.

At a high level, deep neural networks are stacks of nonlinear operations, typically with millions of parameters. This produces a highly flexible and powerful model which has proved effective in many applications. The design of network architectures and optimization methods have been the focus of intense research.

• Course overview
• Pre-requisites
• Homeworks

Topics include:

• convolution neural networks
• recurrent neural networks
• deep reinforcement learning

Homeworks on image classification, video recognition, and deep reinforcement learning. Training of deep learning models using TensorFlow and PyTorch. A large amount of GPU resources are provided to the class.

Mathematical analysis of neural networks, reinforcement learning, and stochastic gradient descent algorithms will also be covered in lectures. (However, there will be no proofs in homeworks and the midterm.)

IE 534 Deep Learning is cross-listed with CS 598. This course is part of the Deep Learning sequence:

• IE 398 Deep Learning (undergraduate version)
• IE 534 Deep Learning
• IE 598 Deep Learning II

Probability, Linear Algebra, and proficiency in Python. CS446 or equivalent courses.

• HW1: Implement and train a neural network from scratch in Python for the MNIST dataset (no PyTorch).
• HW2: Implement and train a convolution neural network from scratch in Python for the MNIST dataset (no PyTorch).
• HW3: Train a deep convolution network on a GPU with PyTorch for the CIFAR10 dataset.
• HW4: Implement a deep residual neural network for CIFAR100.
• HW5: Implement a deep learning model for image ranking.
• HW6: Generative adversarial networks (GANs).
• HW7: Natural Language Processing A.
• HW8: Natural Language Processing B.
• HW9: Video recognition I.