Keras implementation of DQN (DQN.ipynb) for MsPacman-v0 from OpenAI Gym.

Implements Deep Q-network (DQN) in Keras following the architecture proposed in the 2013 paper by V. Mnih et al., "Playing Atari with Deep Reinforcement Learning": arXiv:1312.5602. See: http://www.davidqiu.com:8888/research/nature14236.pdf

The agent learns to play the MsPacman-v0 Gym environment.

I modified the example found from https://keon.io/deep-q-learning/ by implementing the CNN model, target model logic, and frame averaging (among other things).

Hyperparameters were chosen according to the original paper as well as from https://github.com/ageron/tiny-dqn which also provided the image preprocessing method. One key trick I found lead to better policies was introducing a fixed penalty of -1 at each action which did not naturally have a reward.

This is a Python 3 project and you should have a local Python 3 venv.

1. Install libraries: $ pip install -r requirements.txt
2. Run notebooks: $ jupyter notebook

Double Deep Q-network (DDQN.ipynb) is implemented. See: https://arxiv.org/pdf/1509.06461.pdf

• Try different Atari environments
• Experiment with hyperparameters
• Classical algorithm problems (learn functions)