This repository is a PyTorch implementation of the paper "The Option-Critic Architecture" by Pierre-Luc Bacon, Jean Harb and Doina Precup arXiv. It is mostly a rewriting of the original Theano code found here into PyTorch.

Currently, the feature based model can learn CartPole-v0 with a learning rate of 0.005, this has however only been tested with two options. (I dont see any reason to use more than two in the cart pole environment.) the current runs directory holds the training results for this env with 0.005 and 0.006 learning rates.

I suspect it will only take a grid search over learning rate to work on Pong and such. Just supply the right --env argument and the model should switch between features and convolutions.

There are plenty of resources to find a numpy version of the four rooms experiment, this one is a little bit different; represent the state as a one-hot encoded vector, and learn to solve this grid world using a deep net. To enable this experiment, toggle python main.py --switch-goal True --env fourrooms

pytorch 1.3.0
tensorboard 2.0.2
gym 0.15.3

• Using only one optimizer (RMSProp) for both acto and critic.