Make sure there is a set of walkable tiles

Docs links are broken or lead to a nearly empty notebook.

It's a cool project, but is it still maintained or abandoned ?

The obstacle here is that Python might fail to find a correct method order, since all entitites have a position.

For clarity and to minimise entropy, entities should not implement @property, but full members.

For example:

class Foo(Entity):
  walkable: Array = jnp.asarray(False)

While this can be problematic for broadcasting, when entities are batched, the create method should take care of that.

Just a heads up.

Was forwarding this to a friend, and just noticed that the link to the performances notebook is broken in the readme, here:

This is problematic because the wall index position will be converted into an abstract array, or we cannot take mask the rooms as this will require slicing with a dynamic slice.
See https://github.com/epignatelli/navix/blob/dev/keydoorsize/navix/grid.py for example.

The current solution for State is not scalable to other, new entities.

Consider replacing the state set of players doors, keys goals with a single (fixed-length at compile time) collection of entities.

From this:

class State(struct.PyTreeNode):
    """The Markovian state of the environment"""

    key: KeyArray
    """The random number generator state"""
    grid: Array
    """The base map of the environment that remains constant throughout the training"""
    cache: RenderingCache
    """The rendering cache to speed up rendering"""
    players: Player = Player.create()
    """The player entity"""
    goals: Goal = Goal.create()
    """The goal entity, batched over the number of goals"""
    keys: Key = Key.create()
    """The key entity, batched over the number of keys"""
    doors: Door = Door.create()

To this:

class State(struct.PyTreeNode):
    """The Markovian state of the environment"""

    key: KeyArray
    """The random number generator state"""
    grid: Array
    """The base map of the environment that remains constant throughout the training"""
    cache: RenderingCache
    """The rendering cache to speed up rendering"""
    entities: Tuple[Entity, ...]

The main obstacle is the computational cost of iterating though the list when we need to extract a specific entity, like a player for the action, for example.

The current signature of a reward function takes three parameters: the previous state, the action taken in that state, and the following state.
This means that the reward function is not Markovian, which breaks canonical RL assumptions.
See https://arxiv.org/abs/2111.00876 or https://arxiv.org/abs/2212.10420 for me.

The reward function should take only two parameters R(s,a)

Although, we can still control the environment seed with the initial reset(key).

Version is currently uploaded manually before merging to main.

It would be convenient to update that automatically at every pull to main.
Notice that you need to pull again the automatic commit to be up to date with main

For some reason, mkdocs fails to pull/push the latest changes to the gh-pages branch when updating docs.

Source: https://minigrid.farama.org/environments/minigrid/

• MiniGrid-BlockedUnlockPickup-v0
• MiniGrid-LavaCrossingS9N1-v0
• MiniGrid-LavaCrossingS9N2-v0
• MiniGrid-LavaCrossingS9N3-v0
• MiniGrid-LavaCrossingS11N5-v0
• MiniGrid-SimpleCrossingS9N1-v0
• MiniGrid-SimpleCrossingS9N2-v0
• MiniGrid-SimpleCrossingS9N3-v0
• MiniGrid-SimpleCrossingS11N5-v0
• MiniGrid-DistShift1-v0
• MiniGrid-DistShift2-v0
• MiniGrid-DoorKey-5x5-v0
• MiniGrid-DoorKey-6x6-v0
• MiniGrid-DoorKey-8x8-v0
• MiniGrid-DoorKey-16x16-v0
• MiniGrid-Dynamic-Obstacles-5x5-v0
• MiniGrid-Dynamic-Obstacles-Random-5x5-v0
• MiniGrid-Dynamic-Obstacles-6x6-v0
• MiniGrid-Dynamic-Obstacles-Random-6x6-v0
• MiniGrid-Dynamic-Obstacles-8x8-v0
• MiniGrid-Dynamic-Obstacles-16x16-v0
• MiniGrid-Empty-5x5-v0
• MiniGrid-Empty-Random-5x5-v0
• MiniGrid-Empty-6x6-v0
• MiniGrid-Empty-Random-6x6-v0
• MiniGrid-Empty-8x8-v0
• MiniGrid-Empty-16x16-v0
• MiniGrid-Fetch-5x5-N2-v0
• MiniGrid-Fetch-6x6-N2-v0
• MiniGrid-Fetch-8x8-N3-v0
• MiniGrid-FourRooms-v0
• MiniGrid-GoToDoor-5x5-v0
• MiniGrid-GoToDoor-6x6-v0
• MiniGrid-GoToDoor-8x8-v0
• MiniGrid-GoToObject-6x6-N2-v0
• MiniGrid-GoToObject-8x8-N2-v0
• MiniGrid-KeyCorridorS3R1-v0
• MiniGrid-KeyCorridorS3R2-v0
• MiniGrid-KeyCorridorS3R3-v0
• MiniGrid-KeyCorridorS4R3-v0
• MiniGrid-KeyCorridorS5R3-v0
• MiniGrid-KeyCorridorS6R3-v0
• MiniGrid-LavaGapS5-v0
• MiniGrid-LavaGapS6-v0
• MiniGrid-LavaGapS7-v0
• MiniGrid-LockedRoom-v0
• MiniGrid-MemoryS17Random-v0
• MiniGrid-MemoryS13Random-v0
• MiniGrid-MemoryS13-v0
• MiniGrid-MemoryS11-v0
• MiniGrid-MemoryS9-v0
• MiniGrid-MemoryS7-v0
• MiniGrid-MultiRoom-N2-S4-v0
• MiniGrid-MultiRoom-N4-S5-v0
• MiniGrid-MultiRoom-N6-v0
• MiniGrid-ObstructedMaze-1Dl-v0
• MiniGrid-ObstructedMaze-1Dlh-v0
• MiniGrid-ObstructedMaze-1Dlhb-v0
• MiniGrid-ObstructedMaze-2Dl-v0
• MiniGrid-ObstructedMaze-2Dlh-v0
• MiniGrid-ObstructedMaze-2Dlhb-v0
• MiniGrid-ObstructedMaze-1Q-v0
• MiniGrid-ObstructedMaze-2Q-v0
• MiniGrid-ObstructedMaze-Full-v0
• MiniGrid-ObstructedMaze-2Dlhb-v1
• MiniGrid-ObstructedMaze-1Q-v1
• MiniGrid-ObstructedMaze-2Q-v1
• MiniGrid-ObstructedMaze-Full-v1
• MiniGrid-Playground-v0
• MiniGrid-PutNear-6x6-N2-v0
• MiniGrid-PutNear-8x8-N3-v0
• MiniGrid-RedBlueDoors-6x6-v0
• MiniGrid-RedBlueDoors-8x8-v0
• MiniGrid-Unlock-v0
• MiniGrid-UnlockPickup-v0

Reproduce:

import navix as nx

nx.make("Navix-KeyCorridorS3R2-v0", observation_fn=nx.observations.rgb)

Hi!

I have been training ppo agents in navix environments and I keep seeing that agents walk through walls/lava. I'm using version 0.6.14 of Navix. I'm using a PPO agent from PureJaxRL (purejaxrl/ppo_minigrid.py). I train the agent, and then load a network to render the behaviour, and I get the following:

from ppo_minigrid import *
from navix import observations, rewards, terminations, register_env
import navix

config = {
    "LR": 2.5e-4,
    "NUM_ENVS": 16,
    "NUM_STEPS": 256,
    "TOTAL_TIMESTEPS": 2e6,
    "UPDATE_EPOCHS": 1,
    "NUM_MINIBATCHES": 8,
    "GAMMA": 0.99,
    "GAE_LAMBDA": 0.95,
    "CLIP_EPS": 0.2,
    "ENT_COEF": 0.05,
    "VF_COEF": 0.5,
    "MAX_GRAD_NORM": 0.5,
    "ACTIVATION": "tanh",
    "ENV_NAME": "Navix-LavaGapS7-v0",
    "ANNEAL_LR": True,
    "DEBUG": True,
}

rng = jax.random.PRNGKey(30)
train_jit = jax.jit(make_train(config))
out = train_jit(rng)

import matplotlib.pyplot as plt
import jax
import jax.numpy as jnp

env = NavixGymnaxWrapper("Navix-LavaGapS7-v0")
env = FlattenObservationWrapper(env)
env = LogWrapper(env)
key = jax.random.PRNGKey(30)
key, _key = jax.random.split(key)
obs, state = env.reset(_key, None)

def render(obs, title):
    plt.imshow(obs)
    plt.title(title)
    plt.axis('off')
    plt.savefig("image.png")

render(observations.rgb(state.env_state.state), "Initial observation")
network = ActorCritic(env.action_space(None).n, activation=config["ACTIVATION"])
params = out['runner_state'][0].params

for t in range(100):
    rng, _rng = jax.random.split(rng)
    pi, _ = network.apply(params, obs)
    action = pi.sample(seed=_rng)
    print(action)
    obs, state, reward, done, info = env.step(rng, state, action, None)
    render(observations.rgb(state.env_state.state), f"Step {t}")
    input('Continue...')

The initial observation:

After 2 steps:

After 3 steps:

I had the same with the DoorKey environments, where agents simply walk through the dividing wall. Any ideas of what is happening?

Thanks!

Daniel

Maybe add themes?

We can have a minigrid theme, but also a different one.