question about REM baseline implementation about edac HOT 2 CLOSED

Hi. It is worth noting that REM takes a random convex combination of Q-network outputs to compute Q-values and their Bellman targets, while SAC-N takes the minimum of Q-network outputs to compute Bellman targets. Taking the minimum is crucial to achieving good performance in offline RL while preventing the explosion of the Q-values. We ran REM with separate Q-networks (multi-network REM) since it has demonstrated superior results compared to multi-head REM.

from edac.

Thanks!

from edac.