[Question] Reward calculation in HalfCheetah environment about gymnasium HOT 4 CLOSED

The existing implementation uses the actual displacement of the robot In order to calculate the reward forward

Your suggestion approximates displacement by assuming that the velocity would be identical throughout the simulation of the environment step

from gymnasium.

@Kallinteris-Andreas Thanks for the quick response.

Unfortunately, I don't think I understand. So, why when I take the velocity information directly, e.g. immediately after the simulation step using self.data.qvel[0], means that I assume the velocity is constant during the simulation step?

You are probably right because this will explain why I get different velocity values(e.g. X_FINITE_DIFF_VELOCITY != X_VELOCITY_AFTER), but I would expect to receive similar velocity values when I just take the velocity information immediately after the simulation step?

Thanks in advance!

from gymnasium.

Velocity is not constant during a step, MuJoCo is a "linear approximation" simulator

for more information, I would direct you to the MuJoCo simulator documentation

from gymnasium.

Thanks, I will directly start reading the docu more extensively.

from gymnasium.