irll / hippo_gym Goto Github PK

Develop the logic for calculating the budget based on the specified parameters and time steps.
Implement methods to set and retrieve the budget.
Monitor the budget usage and alert when the budget is exceeded.

Whenever you reset LunarLander the score carries on from the last attempt when it should reset as well.

Similar to Minigrid, HCraft does not record episode data output for the example

Implement front-end elements such as start, pause, and other controls for the experiment.
Design and create the visual elements and user interface for the experiment.
Handle user interactions and input from the front-end.

Update README documentation for how to run examples step-by-step guide (and later on how to deploy to AWS once that is fixed), where the data from the example experiments are stored etc

Old docker setup and start.sh file are outdated and need to be updated to include swig as well as setup file structure to allow examples to be ran directly.

Currently with the recent fix on Lunar Lander, the controls on the page don't match the actual in game controls, so a quick fix to the front-end will be required.

Websocket through API Gateway is now working, needs some latency handling and documentation on routes

Handle data sent between the backend and frontend via the websocket.
Implement an API that provides an interface for researchers to specify what data they want to send or receive.
Ensure seamless communication and error handling between the different components.

Connect the external learning algorithm to the Hippo Gym product.
Implement methods to send data from Hippo Gym to the external algorithm.
Receive data from the external algorithm and incorporate it into the Hippo Gym system.

When in a Docker container you aren't able to run hcraft_example or any theoretical example that is using an external graphical display because we haven't configured Docker to allow displays.

When attempting to run hcraft_example you are prompted with the following error:

pygame.error: No available video device

Pygame's inability to find a suitable video device for rendering. This issue typically arises when running Pygame inside a Docker container, since the container doesn't have access to the host system's display.

To solve this issue, you can try running the Docker container with X11 forwarding. This will allow Pygame to use your host's X11 server for rendering.

You'd have to make some adjustments to the Dockerfile and the start.sh script, but be cautious to make it work across ALL devices, because there is a lot of different setup methods for variations in computers (windows, macos, linux) so make it compatible for all types

Currently researchers are having issues deploying to AWS

Currently running MiniGrid does not save any output data to json, save json data to show as an example of what is saved from experiment and use cases.

Hi there, currently I'm designing a research project that will do interactive RL with facial expressions. Since the project needs to be done online, I'm investigating what software I can use. In your publication, you were talking about extending this project. Are there any plans or maybe even a roadmap for something like a webcam support?

Currently when testing master branch, you cannot run the environment due to issues with atari-py and box2d

When attempting to run Lunar Lander example, you are able to run the experiment on locally, but when beginning the experiment, you are presented with the following issue:

• This is was from Docker, similar output if local machine

Process Trial:
Traceback (most recent call last):
  File "/usr/lib/python3.8/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.8/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/robadmin/Desktop/HIPPO_Gym/src/hippogym/trial.py", line 35, in build_and_run
    self.run()
  File "/home/robadmin/Desktop/HIPPO_Gym/src/hippogym/trial.py", line 30, in run
    step.run()
  File "/home/robadmin/Desktop/HIPPO_Gym/src/hippogym/trialsteps/gymstep.py", line 88, in run
    self.send_render()
  File "/home/robadmin/Desktop/HIPPO_Gym/src/hippogym/trialsteps/gymstep.py", line 147, in send_render
    rgb_array = self.env.render(mode="rgb_array")
  File "/home/robadmin/Desktop/HIPPO_Gym/.venv/lib/python3.8/site-packages/gym/core.py", line 329, in render
    return self.env.render(*args, **kwargs)
  File "/home/robadmin/Desktop/HIPPO_Gym/.venv/lib/python3.8/site-packages/gym/wrappers/order_enforcing.py", line 51, in render
    return self.env.render(*args, **kwargs)
  File "/home/robadmin/Desktop/HIPPO_Gym/.venv/lib/python3.8/site-packages/gym/wrappers/env_checker.py", line 53, in render
    return env_render_passive_checker(self.env, *args, **kwargs)
  File "/home/robadmin/Desktop/HIPPO_Gym/.venv/lib/python3.8/site-packages/gym/utils/passive_env_checker.py", line 316, in env_render_passive_checker
    result = env.render(*args, **kwargs)
TypeError: render() got an unexpected keyword argument 'mode'

this is coming from the def send_render(self) method in the gymstep.py file, but this send_render function is being used in MiniGrid so it should be working as expected.

    def send_render(self):
        rgb_array = self.env.render(mode="rgb_array")
        try:
            rgb_array = np.array(rgb_array)
        except:
            raise TypeError("Env render should output a numpy array.")
        render = self.render_window.convert_numpy_array_to_base64(rgb_array)
        self.render_window.update(image=render)

Display a guide beside each live experiment which will fetch the keyboard inputs for the given experiment and display it
Currently Lundar Lander already has this implemented, so implement this for Minigrid example as well

For example:

keyboard_to_value = {
        "ArrowRight": HumanValue.RIGHT,
        "ArrowLeft": HumanValue.LEFT,
        "ArrowUp": HumanValue.UP,
        " ": HumanValue.TOGGLE,
        "c": HumanValue.PICKUP,
        "v": HumanValue.DROP,
        "Enter": HumanValue.END,
    }

Front-end will display:

Controls:

Space to Toggle
C to Pickup
V to Drop

etc

Currently when running a Lunar Lander example experiment, pressing the "restart" button does not actually restart the environment. When ending a episode, you can't press "start" to begin the next episode as well not sure if its a bug, look more into it.