This is a clean and robust Pytorch implementation of PPO on Discrete action space. Here is the result:

All the experiments are trained with same hyperparameters. Other RL algorithms by Pytorch can be found here.

gymnasium==0.29.1  
box2d-py==2.3.5 
numpy==1.26.1
pytorch==2.1.0
tensorboard==2.15.1

python==3.11.5

python main.py

where the default enviroment is CartPole-v1.

python main.py --write False --render True --Loadmodel True --ModelIdex 300000

If you want to train on different enviroments

python main.py --EnvIdex 1

The --EnvIdex can be set to be 0 and 1, where

'--EnvIdex 0' for 'CartPole-v1'  
'--EnvIdex 1' for 'LunarLander-v2'   

if you want train on LunarLander-v2, you need to install box2d-py first.

You can use the tensorboard to visualize the training curve. History training curve is saved at '\runs'

For more details of Hyperparameter Setting, please check 'main.py'

Proximal Policy Optimization Algorithms
Emergence of Locomotion Behaviours in Rich Environments