This is the Deep Learning part of the Unsupervised Learning Course of the Master in High-Performance Computing (SISSA/ICTP)

First Part

• Artificial neural networks
• Train, validate and test a deep learning model
• Convolutional neural networks
• Elementary aspects of unsupervised deep learning models

Second Part

• Introduction to the basics of representation learning
• Autoencoders
• ...

• Alessio Ansuini (First Part)
• Alberto Cazzaniga (Second Part)

Institute: Research and Technology Institute, AREA Science Park)

Notebooks of the First Part are gently made available by Marco Zullich (PhD student in Industrial and Information Engineering at University of Trieste)

Day 1

• The artificial neuron
• Possiblities and limitations of a single neuron
• Linear layer
• Non-linearities
• Fully connected architectures
• Softmax layer
• Cross-entropy loss and the MLE principle

Sources (see below): Michael Nielsen's online book, PyTorch Tutorials

Day 2

• Stochastic gradient descent
• Optimization
• Regularization
• Data augmentation

Sources: Michael Nielsen's online book, PyTorch Tutorials

Day 3

• Convolutional networks basics
• Transfer learning

Michael Nielsen's online book, image kernels, PyTorch Tutorials

Day 4

• What is a "good representation"?
• From Principal Component Analysis to Autoencoders
• Autoencoders: construction, properties and drawbacks

Day 5

TO APPEAR

TO APPEAR

There are excellent free resources to deepen your knowledge on topics such as Deep Learning, Reinforcement Learning and more in general Artificial Intelligence.

Here is a selection of very good ones.

Books for free

• Michael Nielsen

  http://neuralnetworksanddeeplearning.com/

• The Deep Learning Book

  https://www.deeplearningbook.org/

  in a single pdf version

  https://github.com/janishar/mit-deep-learning-book-pdf

• Information Theory, Pattern Recognition, and Neural Networks (Dave McKay)

  http://www.inference.org.uk/mackay/itprnn/book.html

• Probabilistic Machine Learning (Kevin Murphy)

  https://probml.github.io/pml-book/book1.html

• Dive into Deep Learning (Amazon group)

  https://d2l.ai/

Courses for free

• Fast AI (Jeremy Howards)

  Invaluable resource for quickly getting your hands dirt into practical Deep Learning

  https://www.fast.ai/

• Deep learning specialization (auditing is for free) on Coursera (Andrew Ng).

  One of the best resources to learn basic and intermediate concepts.

  (Check the Coursera website for other resources: auditing is sometimes for free, certificates are generally not.)

• Deep unsupervised learning (Pieter Abbeel)

  A glimpse into state-of-the-art research problems.

  https://sites.google.com/view/berkeley-cs294-158-sp20/home

• Deep reinforcement learning (Dave Silver)

  The legendary course of Dave Silver on YouTube

  https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLzuuYNsE1EZAXYR4FJ75jcJseBmo4KQ9-

• Neural network class (Hugo Larochelle)

  After almost 10 years still a very useful resource: crystal clear explanations of an impressive amount of topics, starting from the very basics (I used this a lot!)

  https://www.youtube.com/watch?v=SGZ6BttHMPw&list=PL6Xpj9I5qXYEcOhn7TqghAJ6NAPrNmUBH

• Information Theory, Pattern Recognition, and Neural Networks (Dave McKay) (See also the accompanying book)

  The Lectures of Dave MacKay will accompany you to the study of its beautiful book: on of the most precious resources you will find on this topic.

  Information theory is very relevant in many fields, and particularly in Unsupervised Deep Learning.

  https://www.youtube.com/watch?v=BCiZc0n6COY&list=PLruBu5BI5n4aFpG32iMbdWoRVAA-Vcso6

Websites and Blogs

• Deepmind

  https://deepmind.com https://deepmind.com/blog

• OpenAI

  https://openai.com/

YouTube channels

• Yannick Kilcher's channel

  https://www.youtube.com/channel/UCZHmQk67mSJgfCCTn7xBfew

• Two minutes papers

  https://www.youtube.com/user/keeroyz