Example running screen generation, training, testing, and playing Pong using an OpenAI Gym environment.
Tested with Python 3.8 and NVIDIA GeForce RTX 2070 SUPER GPU.
The version off torch included in the requirements file may need to be changed depending on your setup.
git clone https://github.com/jpbelleau/openaigym_pong.git
cd openaigym_pong
pip install -r requirements.txtA Long short-term memory (LSTM) neural network is used along with two fully connected layers.
Net(
(lstm): LSTM(5120, 5120, batch_first=True)
(fc1): Linear(in_features=5120, out_features=1689, bias=True)
(fcO): Linear(in_features=1689, out_features=3, bias=True)
(relu): ReLU()
)Output nodes are treated as a one-hot encoded array (hold, up, down)
The main components are:
-
pong_screen_gen.py - generates screens files for training and testing
-
pong_runner.py - executes training, testing, and playing a neural network model
-
pong_example.py - shows calling as modules instead of the command line
Generates screen files for training and testing saved per won point. File sizes can vary from 25MB to over 600MB. Thresholds define when to move (up or down) and the wider the spread the better the chance the paddle will stay in the same position. Screens are converted to monochrome and the top and bottom are cropped. Additional screen processing occurs in other scripts and the generation is left at the minimum to allow flexibility to change later.
usage: pong_screen_gen.py [-h] [--savepath SAVEPATH] up_threshold dn_threshold num_games
positional arguments:
up_threshold Threshold (less than) to move up
dn_threshold Threshold (greater than) to move down
num_games Number of games ot play
optional arguments:
-h, --help show this help message and exit
--savepath SAVEPATH Directory to save screen sequences (will create a subfolder per game)Executes training, testing, and playing the neural network model. Screens are further modified by cropping the left and right sides and scaled down by a factor of 2. The main subcommands are: train, test, and play.
usage: pong_runner.py [-h] {train,test,play} ...
positional arguments:
{train,test,play} Mode subcommands
train Training commands
test Testing commands
play Play commands
optional arguments:
-h, --help show this help message and exitTrains a neural network model based on generated Pong screens.
usage: pong_runner.py train [-h] --numpoints NUMPOINTS --loadpath LOADPATH --learnrate LEARNRATE [--savepath SAVEPATH] [--savebasename SAVEBASENAME] [--randfiles] [--forcecpu]
optional arguments:
-h, --help show this help message and exit
--savepath SAVEPATH Directory to save model
--savebasename SAVEBASENAME Base file name for model and optium saves
--randfiles Randomize file selection
--forcecpu Only use CPU
required arguments:
--numpoints NUMPOINTS Number of point files to use for training
--loadpath LOADPATH Directory to load screen sequences
--learnrate LEARNRATE Learning rateTests a neural network model based on generated Pong screens.
usage: pong_runner.py test [-h] --numpoints NUMPOINTS --loadpath LOADPATH --loadmodel LOADMODEL [--randfiles] [--forcecpu]
optional arguments:
-h, --help show this help message and exit
--randfiles Randomize file selection
--forcecpu Only use CPU
required arguments:
--numpoints NUMPOINTS Number of point files to use for testing
--loadpath LOADPATH Directory to load screen sequences
--loadmodel LOADMODEL Full path to model to testPlays Pong using a neural network model.
usage: pong_runner.py play [-h] --numgames NUMGAMES --loadmodel LOADMODEL [--rendergame] [--forcecpu]
optional arguments:
-h, --help show this help message and exit
--rendergame Render the game while playing
--forcecpu Only use CPU
required arguments:
--numgames NUMGAMES Number of games to play
--loadmodel LOADMODEL Full path to model to playExample running Pong screen generation, training, testing, and playing based on calling as modules. Script will need to be modified for your environment.
usage: pong_example.py
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent