[question] Action Smoothing about softlearning HOT 3 CLOSED

Hi Antonin,

the smoothing is in fact currently wrong in the code. We are working to fix it. The correct form should be

beta = sqrt(1 - alpha ** 2) / (1 - alpha)

and

smoothing_latent = alpha * smoothing_latent + (1-alpha) * raw_latents

which is exponential smoothing in the latent space. We prefer smoothing the latents instead of the actions because that does not introduce slowness, and the policy can react immediately to a change in the state. On the downside, if the observations (or state) is noisy, smoothing in the latent space can still result lots of noise in the action space.

from softlearning.

Hi,
Thanks for your answer, that makes more sense.
Btw, i would like to congratulate you and your colleagues for this great algorithm. I have been using it recently on several projects (e.g. https://github.com/araffin/learning-to-drive-in-5-minutes) and it works pretty well with only minor hyperparameters tuning.

from softlearning.

The action (latent) smoothing is fixed in #13.

from softlearning.