This project houses all of the code for ProLoNets: Neural-encoding Human Experts' Domain Knowledge to Warm Start Reinforcement Learning.
I've gone ahead and made two separate virtualenvs for the OpenAI gym environments and the StarCraft II environments, both of which are built on Python 3.6. In order to work with the SC2 environments, you must have Python >= 3.6, and then installing the requirements in the sc2_requirements.txt
file should do it. For the gym environments, any Python that works with OpenAI Gym should work, but I haven't tested this.
All of the code to run various domains lives in the runfiles/
directory.
All file involve a few command line arguments, which I'll review now:
-a
or--agent_type
: Which agent should play through the domain. Details below. Default:fc
-e
or--episodes
: How many episodes to run for. Default: 1000-p
or--processes
: How many concurrent processes to run. Default: 1-s
or--sl_init
: Should the agent be trained via imitation learning first? Only applies ifagent_type
isfc
.Default: False-adv
or--adversary
: Should the agent be mistakenly initialized? Only applies ifagent_type
isshallow_prolo
. Default: False
For the -a
or --agent_type
flag, valid options are:
prolo
for a normal ProLoNet agentshallow_prolo
for a non-deepening ProLoNet agentrandom
for a randomly-initialized ProLoNet agentfc
for a fully-connected agentlstm
for an LSTM agent
This file runs both of the OpenAI gym domains from the paper, namely cart pole and lunar lander. It has one additional command line argument:
-env
or--env_type
: Which environment to run. Valid options arecart
andlunar
. Default:cart
In order to run, you must have the a Python environment with the OpenAI Gym installed. Furthermore, you must have box2d-py if you want the lunar lander agents to run. Thegym_requirements.txt
file should have everything necessary for a Python 3.6 environment.
Running a ProLoNet agent on lunar lander for 1500 episodes with 4 concurrent processes looks like:
python gym_runner.py -a prolo -e 1500 -p 4 -env lunar
For the LOKI agent:
python gym_runner.py -a fc -e 1500 -p 4 -env lunar -s True
And for the N-Mistake agent:
python gym_runner.py -a shallow_prolo -e 1500 -p 4 -env lunar -adv True
This file runs the FindAndDefeatZerglings minigame from the SC2LE. Running this is exactly the same as the gym_runner.py
runfile, with the exception that no --env_type
flag exists for this domain. You must also have all of the StarCraft II setup complete, which means having a valid copy of StarCraft II, having Python >= 3.6, and installing the requirements from the sc2_requirements.txt
file. For information on setting up StarCraft II, refer to Blizzard's Documentation and for the minigame itself, you'll need the map from DeepMind's repo.
Running a ProLoNet agent:
python minigame_runner.py -a prolo -e 2000 -p 1
And a fully-connected agent:
python minigame_runner.py -a fc -e 2000 -p 1
And an LSTM agent:
python minigame_runner.py -a lstm -e 2000 -p 1
This file runs the full SC2 game against in-game AI. In game AI difficulty is set on lines 836-838. Simply changing "Difficult.VeryEasy" to "Difficulty.Easy", "Difficulty.Medium", or "Difficulty.Hard" does the trick. Again, you'll need SC2 and all of the requirements for the appropriate Python environment, as discussed above. Running a ProLoNet agent:
python sc_runner.py -a prolo -e 500 -p 1
And a random ProLoNet agent:
python sc_runner.py -a random -e 500 -p 1
And a non-deepening ProLoNet agent:
python sc_runner.py -a shallow_prolo -e 500 -p 1
Any questions about the work or the code can be either opened up as GitHub issues, or sent directly to me at [email protected]