r2d2 atari about di-engine HOT 8 CLOSED

Hello @hlsafin, first of all you should make sure to train your agent for enough training steps in order to see meaningful results. For example, using our default configuration, R2D2 takes about 4M steps to converge on Pong and 10M to reach good performance on Spaceinvaders. Moreover, there are two important h-paramerers you may want to tune for different environments:

• burnin_step: how many steps to use for initializing RNN hidden state, since those steps are not used for training, you could reduce them in those envs where initial steps are the most important.
• learn_unroll_len: length of a trajectory, you could increase this value when having a longer memory is critical.

You can refer to this file to see all the h-parameters that can be tuned: https://github.com/opendilab/DI-engine/blob/main/dizoo/atari/config/serial/pong/pong_r2d2_config.py

Following, I'm going to attach our training log relative to the Pong environment so that you can make a comparison.

from di-engine.

Yeah, that's so weird, I am running the same file as you, but getting different results. Hmm

from di-engine.

at 2.4 million, the mean reward is still around -20

from di-engine.

can you upload your tensorboard event?

from di-engine.

yeah, I don't think I can upload files here. Maybe I can email you? I tried oppo_config, and it converged. Just no luck with r2d2 config.py

from di-engine.

yeah, I don't think I can upload files here. Maybe I can email you? I tried oppo_config, and it converged. Just no luck with r2d2 config.py

You can send email to our email or contact us on slack channel

from di-engine.

yeah, I can't seem to find it. When you ran your config, did you make any changes to the config file? Also, which version are you using?

from di-engine.

slack channel link: https://join.slack.com/t/opendilab/shared_invite/zt-v9tmv4fp-nUBAQEH1_Kuyu_q4plBssQ

from di-engine.