A Simpler Configuration for Searing Spotlights about endless-memory-gym HOT 2 CLOSED

Hi @subho406,

Great to hear that you enjoy this project!

Searing Spotlights is a tricky one. In the ICLR paper, the recurrent PPO agent was unsuccessful even under ground truth. The good news is, I finally found the culprit for failing at this task. Normalizing the advantages, as part of PPO's objective, makes training highly unstable. If you are training a PPO agent, don't normalize advantages.

Another aid to the agent is to add a reconstruction loss to the agent. This way, the agent needs to adopt an autoencoder target by reconstructing the fed observation.

Concerning simplifying Searing Spotlights, I can't think of any suitable simplification that would maintain the memory dependence as well as the task itself.

Here is a website showing multiple agent behaviors on all environments:
https://marcometer.github.io/
The website is still being developed, and a related journal paper is also in progress. I hope to have it ready in two weeks.

from endless-memory-gym.

I'm closing this now. Let me know if you have further questions.
The mentioned journal paper is finally available as preprint.
https://arxiv.org/abs/2309.17207

from endless-memory-gym.