aowen87 / ppo_and_friends Goto Github PK

Let's add the remaining MuJoCo environments.

Let's convert docstrings to a more conventional pythonic method. I think the following is very clean and easy to read for functions/methods:

def foo(arg1, arg2):
    """
    Description of foo.

    Parameters
    -----------
    arg1: type
        description of arg1
    arg2: type
        description of arg2

    Returns
    --------
    type
        Description of return
    """
    ...
    return val

Let's add the robotics environments.

I should set up a simple CI to help avoid introducing bugs.

Currently, we parallelize across both environments and models. Let's look into adding an option to parallelize over environments while keeping a centralized model that accepts batches of parallelized data.

There were some very large changes recently. Let's make sure all environments are still training optimal policies.

Let's add multi-processing support in two forms:

1. MPI for HPC distributed systems (ideally with GPUs)
2. CPU threading for non-GPU systems

Lots of doc strings are still missing.

Let's add the ability to handle multi-agent environments using MAPPO.

Let's add an option to save out our average rewards for plotting. This will help with determining convergence.

It looks like I broke lstm compatibility with my batch aug update.

It looks like the top score has gone up recently, but the average over 100 test runs has dropped due to instability. Possible causes I can think of: change to advantage normalization; distribution action updates.