Is there a way to do a quick Atari benchmark test with each model? about mushroom-rl HOT 3 CLOSED

Yes, you can use this example:
https://github.com/MushroomRL/mushroom-rl/blob/dev/examples/atari_dqn.py
you have just to change the DQN/DDQN/similar algorithms with your implementation.
You will need to change the code, particularly at lines 328-363 where the algorithms are constructed.

If you are talking about mushroom-rl-benchmarking, unfortunately, I have again to say that it's still not quite ready.

If you need a better explanation of the code, you can look at the simplified version in the tutorial:
https://mushroomrl.readthedocs.io/en/latest/source/tutorials/tutorials.3_deep.html

from mushroom-rl.

I mean more in terms of hyperparams and changing the algorithm. Is it easy to configure other algorithms for Atari?

from mushroom-rl.

The hyperparameter search toolbox is not part of mushroom at all. You can combine Mushroom structure with other optimizers (e.g. Optuna) to achieve this goal. But we don't give any support on this aspect, by the design of MushroomRL.

you can MushroomRL Benchmark to:

• run standard benchmarks to get a measure of the implementation of mushroom algorithms
• create new benchmarks and compare new algorithms with the existing ones.
  It is not a tool to do RL research (if you exclude the research in the benchmarking field).

If you want to use Mushroom for your research, we suggest having a look at:

• the tutorials (have a look at the four tutorials on the left)
• the examples, maybe starting from the more simple ones such as the simple chain, the plotting and normalization and the acrobot.
• Finally, you can look at the atari linked above, which can be the starting point of your work.

Remember that the idea of Mushroom is to provide a clean, organized, and extensible platform to do RL research. By design, it is not a tool to launch easily parameter searches with a configuration file. Other libraries are better suited for this task.
With Mushroom, it is easier to develop new algorithms and reuse existing code in your novel approaches.

from mushroom-rl.