• python version 3.9.16
• PyYAML (yaml parser)
• NumPy (high-performance matrix library)
• PyTorch, aka “torch” (tensor & neural network library)
• OpenAI Gym version 0.25.2, aka “gym” (RL environment simulation library)
• Pygame & imageio-ffmpeg (for OpenAI video rendering)
• scikit-learn (sklearn)
• Matplotlib (data visualization)

• Hw1: Rabbit farm
  To make you be familiar with python and its libraries.

• Hw2: Value Function using matrix approach
  Numpy is a good tools for solving matrix computing problems.

• Hw3: Value Function & optimal policy using iterative DP method
  Getting to know more about RL with "Values" & "Policy".

• Hw4: Monte Carlo with Cat-Mouse environment
  Testing environments with Cat-Mouse envs with our First real RL method "Monte Carlo".

• Hw5: Q-Learning with Cat-Mouse env
  A "model free" method, computing "Q(S,A)" without building the States as last homework.

• Hw6: Sarsa & Expected Sarsa with epsilon greedy policy
  Sarsa is a little bit more conservative strategy with more exploring state while implementing "Epsilon Decay".

• Hw7: Pytorch & CNN
  An overview with CNN models & pytorch API.

• Hw8: Semi-gradient TD(0) with NN function approximation
  Cat-Mouse env again! Learning our "Values" & "Policy" with NN model. Interesting.

• Hw9: Q-Learning with NN && Acrobot environment
  Off-policy Q-Learning sometimes causes "Dead Triad". How do we solve it?

• Hw10: Using Policy-gradient with Monte Carlo to implement the missing code in pg_mc.py

[1] 從根本學習Reinforcement Learning 系列

[2] 强化学习系列（十二）：Eligibility Traces

[3] [Python教學] @property是什麼? 使用場景和用法介紹

[4] Deep Learning 從零開始到放棄的 30 天 PyTorch 數字辨識模型 系列