zela / debugging-dl-models Goto Github PK

• Debug a deep learning network: A nicely written blog post on how to debug a deep learning model.
• Recipe for training neural networks: An Andrej Karpathy's blog post on how to train neural nets.
• Troubleshooting deep learning models: A great video to watch where many debugging steps are summarized in a comprehensive way.
• Machine learning yearning: A practical manual written by Andrew Ng, which gives a full overview how one should structure a deep learning project.
• Bayesian optimization: Bayesian optimization for hyperparameter search.
• Dive into deep learning: An amazing website with theory of deep learning, code examples, exercises. It starts from basics and covers the most advanced topics in DL.
• Deep learning: The Bible of deep learning written by Ian Goodfellow and Yoshua Bengio and Aaron Courville. If you wanna go deep, this book is a must to read.

• Understanding LSTM networks: A clear written post on LSTMs. Good for a quick overview and recalling some basics.
• Batch normalization explained: A blog post explaining batch normalization in detail.
• See-RNN package: A package that helps to visualize RNNs. Check it out if you want to dig into RNNs.
• Gradient clipping: An article on gradient clipping in RNNs.
• Axiomatic attribution for deep networks: An article about Integrated Gradients, which is a useful tool for debugging neural nets.
• Attribution baselines: A really well written blog post on importance of choosing a good baseline for Integrated Gradients.
• Teachin Large Language Models to Self-Debug: A paper about Rubber Duck self-debugging with LLMs.
• LangChain Docs: LangChain documentation page.

• Clone the repo:

git clone https://github.com/stanislav-chekmenev/debugging-dl-models

• Please, create a new virtual environment with Python=3.10 inside code_debugging directory. Feel free to use any of your choice. I prefer venv.

Conda:

conda create --name <name> python=3.10
conda activate <name>

Virtual environment: For Ubuntu 22.04:

python3 --version # make sure you have python 3.10 installed
sudo apt install python3-venv
python3 -m venv <path/to/venv>
source <path/to/venv>/bin/activate

• Upgrade pip, it might be of an old version

pip install pip --upgrade

• Install requirements:

pip install -r requirements.txt

• Install VS Code:

  • Follow this link for the installation details.
  • If you are on Linux Ubuntu starting from 16.04, then please use the following command:

   sudo snap install --classic code

• Create a new project and choose the ealier created virtual env in VS Code:

  • Start VS Code and choose the folder for the project where you cloned the repo.

• Colab 0: Introduction to debugging deep learning models
• Colab 1: Implementation recipe of a bug free model
• Colab 2: Most common bugs
• Colab 3: Rubber duck debugging with LLMs