Comments (47)
pseudo-rnd-thoughts
commented on September 13, 2024
1
@HernandezEduin also, if you have some example actions list for a fixed seed for the MuJoCo team to use, it would be helpful
from gymnasium.
HernandezEduin
commented on September 13, 2024
1
Okay, so I've tested it out with various versions of Mujoco and it the results are as follows:
mujoco==2.3.6 - working
mujoco==2.3.7 - working
mujoco==3.0.0 - not working
mujoco==3.0.1 - not working
mujoco==3.1.0 - not working
mujoco==3.1.1 - not working
mujoco==3.1.2 - not working
mujoco==3.1.3 - not working
The collisions stopped working with Ver. 3.0.0. I'll report it soon to the Mujoco team.
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
1
I agree with the suggested change for version 4.
Normally, i would not agree with you, about changing the version 5 pusher, but because the bug is so critical and it was released in an alpha version of gymnasium we should fix it.
from gymnasium.
pseudo-rnd-thoughts
commented on September 13, 2024
1
@HernandezEduin I'm guessing you using the pretrained model to test with this. Once we have the final xml model complete, it would be helpful to retrain the agent for scratch on the new model and check that the agent will learn the correct model
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
- does this also happen with
mujoco==2.3.7 - can you produce the video with a different angle
from gymnasium.
HernandezEduin
commented on September 13, 2024
- Yes, same issue with Mujoco==2.3.7.
- Is there a way to modify the angle with the Gymnasium record_video?
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
Yes, you need to change the azimuth elevation camera properties
import gymnasium as gym
import numpy as np
DEFAULT_CAMERA_CONFIG = {
"trackbodyid": -1,
#"distance": 4.0,
"distance": 3.0,
"azimuth": 135.0,
"elevation": -22.5,
}
env = gym.make("Pusher-v5", default_camera_config=DEFAULT_CAMERA_CONFIG, render_mode="human")from gymnasium.
HernandezEduin
commented on September 13, 2024
- The changing of camera option is not present for Gymnasium 0.29.1. If I upgrade to gymnasium==1.0.0a1, the option becomes available along with Pusher-v5, but I'm no longer able to run CleanRL due to compatibilities issues of this version of Gym and Stable-Baselines3.
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
You could try this:
import gymnasium
DEFAULT_CAMERA_CONFIG = {
"trackbodyid": -1,
#"distance": 4.0,
"distance": 3.0,
"azimuth": 135.0,
"elevation": -22.5,
}
env = gymnasium.make("Pusher-v4", render_mode="human")
env.unwrapped.mujoco_renderer.default_cam_config = DEFAULT_CAMERA_CONFIG
env.unwrapped.mujoco_renderer._set_cam_config()from gymnasium.
HernandezEduin
commented on September 13, 2024
Okay, it seems to be working after I changed the camera angle for some unknown reason.
rl-video-episode-512.mp4
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
Are you using the same policy?
double check: does this also happen for mujoco==3.1.2
double check: does this work without env.unwrapped.mujoco_renderer._set_cam_config()
from gymnasium.
HernandezEduin
commented on September 13, 2024
Actually, I have to run it without env.unwrapped.mujoco_renderer._set_cam_config(), otherwise it gives me an assertion error on self.viewer is not None. As for whether it works with Mujoco==3.1.2, no. It goes right through the object it seems that the version is causing some errors.
With Mujoco==2.3.7
rl-video-episode-8.mp4
With Mujoco==3.1.2
rl-video-episode-8.mp4
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
also try mujoco==3.1.3 if it does not fix the issue
please find which version of mujoco broke it and report it to the mujoco team (https://github.com/google-deepmind/mujoco) also (tag me in the issue)
Thanks
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
Continuing from the mujoco issue
Can you render with contact forces? @HernandezEduin
import gymnasium
import mujoco
env = gymnasium.make("Pusher-v4", render_mode="rgb_array")
env.reset()
env.render()
env.unwrapped.mujoco_renderer.viewer.vopt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTFORCE] = True
env.unwrapped.mujoco_renderer.viewer.vopt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTPOINT] = True
... # run and render your episode here
Thanks!
from gymnasium.
pseudo-rnd-thoughts
commented on September 13, 2024
@Kallinteris-Andreas given that dm-mujoco don't believe that this is a bug on their end and this is actually just an issue with the environment model.
Does this problem continues in v5, if so we fix it in v5 before the full release of v1.0.0?
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
@pseudo-rnd-thoughts The problem is that version 5 of the environments has already been released in 1.0.0a1 (Users might already have started using it in their research ), This should be fixed in a potential version 6 In the meantime, we should block the instantiation of the pusher environment When the MMujoco version is 3.0 or higher
from gymnasium.
pseudo-rnd-thoughts
commented on September 13, 2024
I think we should put a restriction on pusher-v4 for mujoco < 3.0 at the very least as we know this is bugged.
For v5, given that 1.0a requires users to intentionally install it, pip install gymnasium will install 0.29 then I don't think it is an issue to update the model currently. But it would be great to do before alpha 2
Do you think that it would be easy to get a better model?
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
@HernandezEduin can you also test this model file?
pusher_v5.xml
<mujoco model="arm3d">
<compiler inertiafromgeom="true" angle="radian" coordinate="local"/>
<option timestep="0.01" gravity="0 0 0" iterations="20" integrator="Euler" />
<default>
<joint armature='0.04' damping="1" limited="true"/>
<geom friction=".8 .1 .1" density="300" margin="0.002" condim="1" contype="0" conaffinity="0"/>
</default>
<worldbody>
<light diffuse=".5 .5 .5" pos="0 0 3" dir="0 0 -1"/>
<geom name="table" type="plane" pos="0 0.5 -0.325" size="1 1 0.1" contype="1" conaffinity="1"/>
<body name="r_shoulder_pan_link" pos="0 -0.6 0">
<geom name="e1" type="sphere" rgba="0.6 0.6 0.6 1" pos="-0.06 0.05 0.2" size="0.05" />
<geom name="e2" type="sphere" rgba="0.6 0.6 0.6 1" pos=" 0.06 0.05 0.2" size="0.05" />
<geom name="e1p" type="sphere" rgba="0.1 0.1 0.1 1" pos="-0.06 0.09 0.2" size="0.03" />
<geom name="e2p" type="sphere" rgba="0.1 0.1 0.1 1" pos=" 0.06 0.09 0.2" size="0.03" />
<geom name="sp" type="capsule" fromto="0 0 -0.4 0 0 0.2" size="0.1" />
<joint name="r_shoulder_pan_joint" type="hinge" pos="0 0 0" axis="0 0 1" range="-2.2854 1.714602" damping="1.0" />
<body name="r_shoulder_lift_link" pos="0.1 0 0">
<geom name="sl" type="capsule" fromto="0 -0.1 0 0 0.1 0" size="0.1" />
<joint name="r_shoulder_lift_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-0.5236 1.3963" damping="1.0" />
<body name="r_upper_arm_roll_link" pos="0 0 0">
<geom name="uar" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_upper_arm_roll_joint" type="hinge" pos="0 0 0" axis="1 0 0" range="-1.5 1.7" damping="0.1" />
<body name="r_upper_arm_link" pos="0 0 0">
<geom name="ua" type="capsule" fromto="0 0 0 0.4 0 0" size="0.06" />
<body name="r_elbow_flex_link" pos="0.4 0 0">
<geom name="ef" type="capsule" fromto="0 -0.02 0 0.0 0.02 0" size="0.06" />
<joint name="r_elbow_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-2.3213 0" damping="0.1" />
<body name="r_forearm_roll_link" pos="0 0 0">
<geom name="fr" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_forearm_roll_joint" type="hinge" limited="true" pos="0 0 0" axis="1 0 0" damping=".1" range="-1.5 1.5"/>
<body name="r_forearm_link" pos="0 0 0">
<geom name="fa" type="capsule" fromto="0 0 0 0.291 0 0" size="0.05" />
<body name="r_wrist_flex_link" pos="0.321 0 0">
<geom name="wf" type="capsule" fromto="0 -0.02 0 0 0.02 0" size="0.01" />
<joint name="r_wrist_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-1.094 0" damping=".1" />
<body name="r_wrist_roll_link" pos="0 0 0">
<joint name="r_wrist_roll_joint" type="hinge" pos="0 0 0" limited="true" axis="1 0 0" damping="0.1" range="-1.5 1.5"/>
<body name="tips_arm" pos="0 0 0">
<geom name="tip_arml" type="sphere" pos="0.1 -0.1 0." size="0.01" />
<geom name="tip_armr" type="sphere" pos="0.1 0.1 0." size="0.01" />
</body>
<geom type="capsule" fromto="0 -0.1 0. 0.0 +0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 -0.1 0. 0.1 -0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 +0.1 0. 0.1 +0.1 0." size="0.02" contype="1" conaffinity="1" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
<!--<body name="object" pos="0.55 -0.3 -0.275" >-->
<body name="object" pos="0.45 -0.05 -0.275" >
<geom rgba="1 1 1 0" type="sphere" size="0.05 0.05 0.05" density="0.01" conaffinity="0"/>
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.01" contype="1" conaffinity="0"/>
<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
<body name="goal" pos="0.45 -0.05 -0.3230">
<geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/>
<joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
</worldbody>
<actuator>
<motor joint="r_shoulder_pan_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_shoulder_lift_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_upper_arm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_elbow_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_forearm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true"/>
</actuator>
</mujoco>from gymnasium.
HernandezEduin
commented on September 13, 2024
Sorry for the delay! This is the result for Mujoco == 3.1.3 using Gymnasium == 0.29.1.
mujoco.3.1.3.with.Contact.Forces.mp4
Continuing from the
mujocoissueCan you render with contact forces? @HernandezEduin
import gymnasium import mujoco env = gymnasium.make("Pusher-v4", render_mode="rgb_array") env.reset() env.render() env.unwrapped.mujoco_renderer.viewer.vopt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTFORCE] = True env.unwrapped.mujoco_renderer.viewer.vopt.flags[mujoco.mjtVisFlag.mjVIS_CONTACTPOINT] = True ... # run and render your episode hereThanks!
from gymnasium.
HernandezEduin
commented on September 13, 2024
@HernandezEduin can you also test this model file?
pusher_v5.xml<mujoco model="arm3d"> <compiler inertiafromgeom="true" angle="radian" coordinate="local"/> <option timestep="0.01" gravity="0 0 0" iterations="20" integrator="Euler" /> <default> <joint armature='0.04' damping="1" limited="true"/> <geom friction=".8 .1 .1" density="300" margin="0.002" condim="1" contype="0" conaffinity="0"/> </default> <worldbody> <light diffuse=".5 .5 .5" pos="0 0 3" dir="0 0 -1"/> <geom name="table" type="plane" pos="0 0.5 -0.325" size="1 1 0.1" contype="1" conaffinity="1"/> <body name="r_shoulder_pan_link" pos="0 -0.6 0"> <geom name="e1" type="sphere" rgba="0.6 0.6 0.6 1" pos="-0.06 0.05 0.2" size="0.05" /> <geom name="e2" type="sphere" rgba="0.6 0.6 0.6 1" pos=" 0.06 0.05 0.2" size="0.05" /> <geom name="e1p" type="sphere" rgba="0.1 0.1 0.1 1" pos="-0.06 0.09 0.2" size="0.03" /> <geom name="e2p" type="sphere" rgba="0.1 0.1 0.1 1" pos=" 0.06 0.09 0.2" size="0.03" /> <geom name="sp" type="capsule" fromto="0 0 -0.4 0 0 0.2" size="0.1" /> <joint name="r_shoulder_pan_joint" type="hinge" pos="0 0 0" axis="0 0 1" range="-2.2854 1.714602" damping="1.0" /> <body name="r_shoulder_lift_link" pos="0.1 0 0"> <geom name="sl" type="capsule" fromto="0 -0.1 0 0 0.1 0" size="0.1" /> <joint name="r_shoulder_lift_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-0.5236 1.3963" damping="1.0" /> <body name="r_upper_arm_roll_link" pos="0 0 0"> <geom name="uar" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" /> <joint name="r_upper_arm_roll_joint" type="hinge" pos="0 0 0" axis="1 0 0" range="-1.5 1.7" damping="0.1" /> <body name="r_upper_arm_link" pos="0 0 0"> <geom name="ua" type="capsule" fromto="0 0 0 0.4 0 0" size="0.06" /> <body name="r_elbow_flex_link" pos="0.4 0 0"> <geom name="ef" type="capsule" fromto="0 -0.02 0 0.0 0.02 0" size="0.06" /> <joint name="r_elbow_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-2.3213 0" damping="0.1" /> <body name="r_forearm_roll_link" pos="0 0 0"> <geom name="fr" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" /> <joint name="r_forearm_roll_joint" type="hinge" limited="true" pos="0 0 0" axis="1 0 0" damping=".1" range="-1.5 1.5"/> <body name="r_forearm_link" pos="0 0 0"> <geom name="fa" type="capsule" fromto="0 0 0 0.291 0 0" size="0.05" /> <body name="r_wrist_flex_link" pos="0.321 0 0"> <geom name="wf" type="capsule" fromto="0 -0.02 0 0 0.02 0" size="0.01" /> <joint name="r_wrist_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-1.094 0" damping=".1" /> <body name="r_wrist_roll_link" pos="0 0 0"> <joint name="r_wrist_roll_joint" type="hinge" pos="0 0 0" limited="true" axis="1 0 0" damping="0.1" range="-1.5 1.5"/> <body name="tips_arm" pos="0 0 0"> <geom name="tip_arml" type="sphere" pos="0.1 -0.1 0." size="0.01" /> <geom name="tip_armr" type="sphere" pos="0.1 0.1 0." size="0.01" /> </body> <geom type="capsule" fromto="0 -0.1 0. 0.0 +0.1 0" size="0.02" contype="1" conaffinity="1" /> <geom type="capsule" fromto="0 -0.1 0. 0.1 -0.1 0" size="0.02" contype="1" conaffinity="1" /> <geom type="capsule" fromto="0 +0.1 0. 0.1 +0.1 0." size="0.02" contype="1" conaffinity="1" /> </body> </body> </body> </body> </body> </body> </body> </body> </body> <!--<body name="object" pos="0.55 -0.3 -0.275" >--> <body name="object" pos="0.45 -0.05 -0.275" > <geom rgba="1 1 1 0" type="sphere" size="0.05 0.05 0.05" density="0.01" conaffinity="0"/> <geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.01" contype="1" conaffinity="0"/> <joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/> <joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/> </body> <body name="goal" pos="0.45 -0.05 -0.3230"> <geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/> <joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/> <joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/> </body> </worldbody> <actuator> <motor joint="r_shoulder_pan_joint" ctrlrange="-2.0 2.0" ctrllimited="true" /> <motor joint="r_shoulder_lift_joint" ctrlrange="-2.0 2.0" ctrllimited="true" /> <motor joint="r_upper_arm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" /> <motor joint="r_elbow_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" /> <motor joint="r_forearm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" /> <motor joint="r_wrist_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" /> <motor joint="r_wrist_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true"/> </actuator> </mujoco>
The updated xml file seems to have solved the issue!
mujoco.3.1.3.with.updated.pusher.xml.mp4
from gymnasium.
HernandezEduin
commented on September 13, 2024
Actually, it seems it still has the problem... It goes through the object occasionally...
mujoco.3.1.3.with.updated.pusher.problem.mp4
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
- the
objectappears to be a lot heavier and harder to push (from your first video), so this might need an extra revision - it appears that the fingers of the pusher arm tips have no collision
from gymnasium.
HernandezEduin
commented on September 13, 2024
- The density of the object being pushed was 10^-5 and it is now 10^-2.
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
Here is a new revision of the pusher model that should at least fix the first problem,
also, please use width=1280, height=720 in make() arguments, so that the generated video is of a higher resolution
Thanks!
<!--
Pusher model for `Pusher-v5` RC2, based on openai/gym/Pusher
modified by @kallinteris-Andreas
- disabled body.object shephere geometry
- reduced body.object cylinder density 0.00001 -> 0.001
-->
<mujoco model="arm3d">
<compiler inertiafromgeom="true" angle="radian" coordinate="local"/>
<option timestep="0.01" gravity="0 0 0" iterations="20" integrator="Euler" />
<default>
<joint armature='0.04' damping="1" limited="true"/>
<geom friction=".8 .1 .1" density="300" margin="0.002" condim="1" contype="0" conaffinity="0"/>
</default>
<worldbody>
<light diffuse=".5 .5 .5" pos="0 0 3" dir="0 0 -1"/>
<geom name="table" type="plane" pos="0 0.5 -0.325" size="1 1 0.1" contype="1" conaffinity="1"/>
<body name="r_shoulder_pan_link" pos="0 -0.6 0">
<geom name="e1" type="sphere" rgba="0.6 0.6 0.6 1" pos="-0.06 0.05 0.2" size="0.05" />
<geom name="e2" type="sphere" rgba="0.6 0.6 0.6 1" pos=" 0.06 0.05 0.2" size="0.05" />
<geom name="e1p" type="sphere" rgba="0.1 0.1 0.1 1" pos="-0.06 0.09 0.2" size="0.03" />
<geom name="e2p" type="sphere" rgba="0.1 0.1 0.1 1" pos=" 0.06 0.09 0.2" size="0.03" />
<geom name="sp" type="capsule" fromto="0 0 -0.4 0 0 0.2" size="0.1" />
<joint name="r_shoulder_pan_joint" type="hinge" pos="0 0 0" axis="0 0 1" range="-2.2854 1.714602" damping="1.0" />
<body name="r_shoulder_lift_link" pos="0.1 0 0">
<geom name="sl" type="capsule" fromto="0 -0.1 0 0 0.1 0" size="0.1" />
<joint name="r_shoulder_lift_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-0.5236 1.3963" damping="1.0" />
<body name="r_upper_arm_roll_link" pos="0 0 0">
<geom name="uar" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_upper_arm_roll_joint" type="hinge" pos="0 0 0" axis="1 0 0" range="-1.5 1.7" damping="0.1" />
<body name="r_upper_arm_link" pos="0 0 0">
<geom name="ua" type="capsule" fromto="0 0 0 0.4 0 0" size="0.06" />
<body name="r_elbow_flex_link" pos="0.4 0 0">
<geom name="ef" type="capsule" fromto="0 -0.02 0 0.0 0.02 0" size="0.06" />
<joint name="r_elbow_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-2.3213 0" damping="0.1" />
<body name="r_forearm_roll_link" pos="0 0 0">
<geom name="fr" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_forearm_roll_joint" type="hinge" limited="true" pos="0 0 0" axis="1 0 0" damping=".1" range="-1.5 1.5"/>
<body name="r_forearm_link" pos="0 0 0">
<geom name="fa" type="capsule" fromto="0 0 0 0.291 0 0" size="0.05" />
<body name="r_wrist_flex_link" pos="0.321 0 0">
<geom name="wf" type="capsule" fromto="0 -0.02 0 0 0.02 0" size="0.01" />
<joint name="r_wrist_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-1.094 0" damping=".1" />
<body name="r_wrist_roll_link" pos="0 0 0">
<joint name="r_wrist_roll_joint" type="hinge" pos="0 0 0" limited="true" axis="1 0 0" damping="0.1" range="-1.5 1.5"/>
<body name="tips_arm" pos="0 0 0">
<geom name="tip_arml" type="sphere" pos="0.1 -0.1 0." size="0.01" />
<geom name="tip_armr" type="sphere" pos="0.1 0.1 0." size="0.01" />
</body>
<geom type="capsule" fromto="0 -0.1 0. 0.0 +0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 -0.1 0. 0.1 -0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 +0.1 0. 0.1 +0.1 0." size="0.02" contype="1" conaffinity="1" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
<!--<body name="object" pos="0.55 -0.3 -0.275" >-->
<body name="object" pos="0.45 -0.05 -0.275" >
<!-- <geom rgba="1 1 1 0" type="sphere" size="0.05 0.05 0.05" density="0.01" conaffinity="0"/> -->
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.001" contype="1" conaffinity="0"/>
<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
<body name="goal" pos="0.45 -0.05 -0.3230">
<geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/>
<joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
</worldbody>
<actuator>
<motor joint="r_shoulder_pan_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_shoulder_lift_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_upper_arm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_elbow_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_forearm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true"/>
</actuator>
</mujoco>from gymnasium.
HernandezEduin
commented on September 13, 2024
rl-video-episode-0.mp4
rl-video-episode-1.mp4
rl-video-episode-8.mp4
The object seems to be pushed very slowly now.
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
does this problem happen for all the episodes?
from gymnasium.
HernandezEduin
commented on September 13, 2024
Yes
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
I re-enabled the sphere collision geometry
<!--
Pusher model for `Pusher-v5` RC3, based on openai/gym/Pusher
modified by @kallinteris-Andreas
- reduced body.object cylinder density 0.00001 -> 0.001
- reduced body.object sphere density 0.00001 -> 0.001
-->
<mujoco model="arm3d">
<compiler inertiafromgeom="true" angle="radian" coordinate="local"/>
<option timestep="0.01" gravity="0 0 0" iterations="20" integrator="Euler" />
<default>
<joint armature='0.04' damping="1" limited="true"/>
<geom friction=".8 .1 .1" density="300" margin="0.002" condim="1" contype="0" conaffinity="0"/>
</default>
<worldbody>
<light diffuse=".5 .5 .5" pos="0 0 3" dir="0 0 -1"/>
<geom name="table" type="plane" pos="0 0.5 -0.325" size="1 1 0.1" contype="1" conaffinity="1"/>
<body name="r_shoulder_pan_link" pos="0 -0.6 0">
<geom name="e1" type="sphere" rgba="0.6 0.6 0.6 1" pos="-0.06 0.05 0.2" size="0.05" />
<geom name="e2" type="sphere" rgba="0.6 0.6 0.6 1" pos=" 0.06 0.05 0.2" size="0.05" />
<geom name="e1p" type="sphere" rgba="0.1 0.1 0.1 1" pos="-0.06 0.09 0.2" size="0.03" />
<geom name="e2p" type="sphere" rgba="0.1 0.1 0.1 1" pos=" 0.06 0.09 0.2" size="0.03" />
<geom name="sp" type="capsule" fromto="0 0 -0.4 0 0 0.2" size="0.1" />
<joint name="r_shoulder_pan_joint" type="hinge" pos="0 0 0" axis="0 0 1" range="-2.2854 1.714602" damping="1.0" />
<body name="r_shoulder_lift_link" pos="0.1 0 0">
<geom name="sl" type="capsule" fromto="0 -0.1 0 0 0.1 0" size="0.1" />
<joint name="r_shoulder_lift_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-0.5236 1.3963" damping="1.0" />
<body name="r_upper_arm_roll_link" pos="0 0 0">
<geom name="uar" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_upper_arm_roll_joint" type="hinge" pos="0 0 0" axis="1 0 0" range="-1.5 1.7" damping="0.1" />
<body name="r_upper_arm_link" pos="0 0 0">
<geom name="ua" type="capsule" fromto="0 0 0 0.4 0 0" size="0.06" />
<body name="r_elbow_flex_link" pos="0.4 0 0">
<geom name="ef" type="capsule" fromto="0 -0.02 0 0.0 0.02 0" size="0.06" />
<joint name="r_elbow_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-2.3213 0" damping="0.1" />
<body name="r_forearm_roll_link" pos="0 0 0">
<geom name="fr" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_forearm_roll_joint" type="hinge" limited="true" pos="0 0 0" axis="1 0 0" damping=".1" range="-1.5 1.5"/>
<body name="r_forearm_link" pos="0 0 0">
<geom name="fa" type="capsule" fromto="0 0 0 0.291 0 0" size="0.05" />
<body name="r_wrist_flex_link" pos="0.321 0 0">
<geom name="wf" type="capsule" fromto="0 -0.02 0 0 0.02 0" size="0.01" />
<joint name="r_wrist_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-1.094 0" damping=".1" />
<body name="r_wrist_roll_link" pos="0 0 0">
<joint name="r_wrist_roll_joint" type="hinge" pos="0 0 0" limited="true" axis="1 0 0" damping="0.1" range="-1.5 1.5"/>
<body name="tips_arm" pos="0 0 0">
<geom name="tip_arml" type="sphere" pos="0.1 -0.1 0." size="0.01" />
<geom name="tip_armr" type="sphere" pos="0.1 0.1 0." size="0.01" />
</body>
<geom type="capsule" fromto="0 -0.1 0. 0.0 +0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 -0.1 0. 0.1 -0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 +0.1 0. 0.1 +0.1 0." size="0.02" contype="1" conaffinity="1" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
<!--<body name="object" pos="0.55 -0.3 -0.275" >-->
<body name="object" pos="0.45 -0.05 -0.275" >
<geom rgba="1 1 1 0" type="sphere" size="0.05 0.05 0.05" density="0.001" conaffinity="0"/>
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.001" contype="1" conaffinity="0"/>
<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
<body name="goal" pos="0.45 -0.05 -0.3230">
<geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/>
<joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
</worldbody>
<actuator>
<motor joint="r_shoulder_pan_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_shoulder_lift_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_upper_arm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_elbow_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_forearm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true"/>
</actuator>
</mujoco>from gymnasium.
HernandezEduin
commented on September 13, 2024
rl-video-episode-0.mp4
rl-video-episode-1.mp4
rl-video-episode-8.mp4
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
Hmm, I have little idea of why this is actually happening :-)
double check you used the correct model file, (the results are identical to the previous case, but that can be explained by the fact that the bug is the same for both cases)
here is RC4 it is identical to RC1, but with the sphere object disabled, if that does not work I guess we can try decreasing the object's density to 0.005?
<!--
Pusher model for `Pusher-v5` RC4, based on openai/gym/Pusher
modified by @kallinteris-Andreas
- reduced body.object cylinder density 0.00001 -> 0.01
- disabled body.object sphere (density 0.00001 -> 0.001)
-->
<mujoco model="arm3d">
<compiler inertiafromgeom="true" angle="radian" coordinate="local"/>
<option timestep="0.01" gravity="0 0 0" iterations="20" integrator="Euler" />
<default>
<joint armature='0.04' damping="1" limited="true"/>
<geom friction=".8 .1 .1" density="300" margin="0.002" condim="1" contype="0" conaffinity="0"/>
</default>
<worldbody>
<light diffuse=".5 .5 .5" pos="0 0 3" dir="0 0 -1"/>
<geom name="table" type="plane" pos="0 0.5 -0.325" size="1 1 0.1" contype="1" conaffinity="1"/>
<body name="r_shoulder_pan_link" pos="0 -0.6 0">
<geom name="e1" type="sphere" rgba="0.6 0.6 0.6 1" pos="-0.06 0.05 0.2" size="0.05" />
<geom name="e2" type="sphere" rgba="0.6 0.6 0.6 1" pos=" 0.06 0.05 0.2" size="0.05" />
<geom name="e1p" type="sphere" rgba="0.1 0.1 0.1 1" pos="-0.06 0.09 0.2" size="0.03" />
<geom name="e2p" type="sphere" rgba="0.1 0.1 0.1 1" pos=" 0.06 0.09 0.2" size="0.03" />
<geom name="sp" type="capsule" fromto="0 0 -0.4 0 0 0.2" size="0.1" />
<joint name="r_shoulder_pan_joint" type="hinge" pos="0 0 0" axis="0 0 1" range="-2.2854 1.714602" damping="1.0" />
<body name="r_shoulder_lift_link" pos="0.1 0 0">
<geom name="sl" type="capsule" fromto="0 -0.1 0 0 0.1 0" size="0.1" />
<joint name="r_shoulder_lift_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-0.5236 1.3963" damping="1.0" />
<body name="r_upper_arm_roll_link" pos="0 0 0">
<geom name="uar" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_upper_arm_roll_joint" type="hinge" pos="0 0 0" axis="1 0 0" range="-1.5 1.7" damping="0.1" />
<body name="r_upper_arm_link" pos="0 0 0">
<geom name="ua" type="capsule" fromto="0 0 0 0.4 0 0" size="0.06" />
<body name="r_elbow_flex_link" pos="0.4 0 0">
<geom name="ef" type="capsule" fromto="0 -0.02 0 0.0 0.02 0" size="0.06" />
<joint name="r_elbow_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-2.3213 0" damping="0.1" />
<body name="r_forearm_roll_link" pos="0 0 0">
<geom name="fr" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_forearm_roll_joint" type="hinge" limited="true" pos="0 0 0" axis="1 0 0" damping=".1" range="-1.5 1.5"/>
<body name="r_forearm_link" pos="0 0 0">
<geom name="fa" type="capsule" fromto="0 0 0 0.291 0 0" size="0.05" />
<body name="r_wrist_flex_link" pos="0.321 0 0">
<geom name="wf" type="capsule" fromto="0 -0.02 0 0 0.02 0" size="0.01" />
<joint name="r_wrist_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-1.094 0" damping=".1" />
<body name="r_wrist_roll_link" pos="0 0 0">
<joint name="r_wrist_roll_joint" type="hinge" pos="0 0 0" limited="true" axis="1 0 0" damping="0.1" range="-1.5 1.5"/>
<body name="tips_arm" pos="0 0 0">
<geom name="tip_arml" type="sphere" pos="0.1 -0.1 0." size="0.01" />
<geom name="tip_armr" type="sphere" pos="0.1 0.1 0." size="0.01" />
</body>
<geom type="capsule" fromto="0 -0.1 0. 0.0 +0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 -0.1 0. 0.1 -0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 +0.1 0. 0.1 +0.1 0." size="0.02" contype="1" conaffinity="1" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
<!--<body name="object" pos="0.55 -0.3 -0.275" >-->
<body name="object" pos="0.45 -0.05 -0.275" >
<!-- <geom rgba="1 1 1 0" type="sphere" size="0.05 0.05 0.05" density="0.001" conaffinity="0"/> -->
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.01" contype="1" conaffinity="0"/>
<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
<body name="goal" pos="0.45 -0.05 -0.3230">
<geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/>
<joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
</worldbody>
<actuator>
<motor joint="r_shoulder_pan_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_shoulder_lift_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_upper_arm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_elbow_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_forearm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true"/>
</actuator>
</mujoco>from gymnasium.
HernandezEduin
commented on September 13, 2024
rl-video-episode-1.mp4
rl-video-episode-8.mp4
rl-video-episode-0.mp4
I copied the previous xml, but instead of decreasing the density for the object, I increased it to 0.5.
from gymnasium.
HernandezEduin
commented on September 13, 2024
rl-video-episode-0.mp4
rl-video-episode-1.mp4
rl-video-episode-8.mp4
More videos of the same, but this time the arms tips colliding.
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
"""
I copied the previous xml, but instead of decreasing the density for the object, I increased it to 0.5.
"""
I am not sure what xml file you are talking about, please use the "RCx" name (found at the top of the files) just post your RC
Thanks!
@pseudo-rnd-thoughts we can move to validation after we have finalized the model
from gymnasium.
HernandezEduin
commented on September 13, 2024
Sorry, I meant I used the RC4 and modified the object's cylinder's density to 0.5.
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.5" contype="1" conaffinity="0"/>
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
The only difference between RC4 and RC2 is the cylinder density
with RC4 having 0.01 and rc2 having 0.001
where with RC4 the object's mass corresponds to 7.85398163e-06 which should be easily moveable by the pusher arm if trained
So I believe we should move to validation
we will need to use the current "Pusher-v5" environment in the 1.0alpha1, but with the new xml_file (RC4).
For training, you can use any state-of-the-art algorithm you want (if you have a fast enough computer you can do multiple algorithms), but it must be a standard implementation from a library like SB3, cleanRL or others, 1mil steps over episodes should be enough.
If this results in the trained policy being able to "solve" the environment, then we can lock in the xml_file, if not we will have to test a new revision model version RC5.
Also after that we will have to define the reward_threshold parameter
@HernandezEduin do you want to do it?
from gymnasium.
HernandezEduin
commented on September 13, 2024
Sure, I've been using the CleanRL library implementation of the TD3 algorithm. I've just been reloading the weights of the model I had trained on the pusher-v3 on mujoco==2.3.7. I'll update my gymnasium to the current version and do the retraining then!
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
Ah wait, you've been using version 3 instead of version 4?
from gymnasium.
HernandezEduin
commented on September 13, 2024
*I meant pusher-v4, sorry
Btw, what version of Stable-Baseline3 is compatible with Gymnasium 1.0alpha1?
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
this one DLR-RM/stable-baselines3#1837
from gymnasium.
HernandezEduin
commented on September 13, 2024
So I've tried to train the pusher-v5 and v4 models using RC4 with various densities for the object, however, it doesn't seem to properly learn as it would with Gymnasium 0.29.1.
Maybe the quick fixes I'm doing on the TD3 file from the cleanrl and the adjustment to the record_video from SB3 aren't appropriate?
Gymnasium == 1.0.0.a1
stable_baselines3 == 2.3.0a4
from gymnasium.
HernandezEduin
commented on September 13, 2024
For reference, I modified the file stable_baselines3\common\vec_env\vec_video_recorder.py as follows
Instead of from gymnasium.wrappers.monitoring import video_recorder, I do from gymnasium.wrappers.rendering import RecordVideo as video_recorder
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
sb3 video record does not work, yet here is what I used:
video_folder = "videos/"
video_length = 1000
VIDEO_NAME = f"{args.env}_{args.algo}_run_{args.run}"
frame_list = []
obs = model.get_env().reset()
for _ in range(video_length + 1):
action, _state = model.predict(obs, deterministic=True)
obs, _, _, _ = model.get_env().step(action)
frame_list.append(model.get_env().render())
model.get_env().close()
from moviepy.video.io.ImageSequenceClip import ImageSequenceClip
import os
clip = ImageSequenceClip(frame_list, fps=eval_env.metadata["render_fps"])
#moviepy_logger = None if self.disable_logger else "bar"
path = os.path.join(video_folder, f"{VIDEO_NAME}.mp4")
clip.write_videofile(path, logger="bar")
#clip.write_videofile(path, logger=moviepy_logger)
exit()from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
<!--
Pusher model for `Pusher-v5` RC5, based on openai/gym/Pusher
modified by @kallinteris-Andreas
- reduced body.object cylinder density 0.00001 -> 0.005
- disabled body.object sphere (density 0.00001 -> 0.001)
-->
<mujoco model="arm3d">
<compiler inertiafromgeom="true" angle="radian" coordinate="local"/>
<option timestep="0.01" gravity="0 0 0" iterations="20" integrator="Euler" />
<default>
<joint armature='0.04' damping="1" limited="true"/>
<geom friction=".8 .1 .1" density="300" margin="0.002" condim="1" contype="0" conaffinity="0"/>
</default>
<worldbody>
<light diffuse=".5 .5 .5" pos="0 0 3" dir="0 0 -1"/>
<geom name="table" type="plane" pos="0 0.5 -0.325" size="1 1 0.1" contype="1" conaffinity="1"/>
<body name="r_shoulder_pan_link" pos="0 -0.6 0">
<geom name="e1" type="sphere" rgba="0.6 0.6 0.6 1" pos="-0.06 0.05 0.2" size="0.05" />
<geom name="e2" type="sphere" rgba="0.6 0.6 0.6 1" pos=" 0.06 0.05 0.2" size="0.05" />
<geom name="e1p" type="sphere" rgba="0.1 0.1 0.1 1" pos="-0.06 0.09 0.2" size="0.03" />
<geom name="e2p" type="sphere" rgba="0.1 0.1 0.1 1" pos=" 0.06 0.09 0.2" size="0.03" />
<geom name="sp" type="capsule" fromto="0 0 -0.4 0 0 0.2" size="0.1" />
<joint name="r_shoulder_pan_joint" type="hinge" pos="0 0 0" axis="0 0 1" range="-2.2854 1.714602" damping="1.0" />
<body name="r_shoulder_lift_link" pos="0.1 0 0">
<geom name="sl" type="capsule" fromto="0 -0.1 0 0 0.1 0" size="0.1" />
<joint name="r_shoulder_lift_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-0.5236 1.3963" damping="1.0" />
<body name="r_upper_arm_roll_link" pos="0 0 0">
<geom name="uar" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_upper_arm_roll_joint" type="hinge" pos="0 0 0" axis="1 0 0" range="-1.5 1.7" damping="0.1" />
<body name="r_upper_arm_link" pos="0 0 0">
<geom name="ua" type="capsule" fromto="0 0 0 0.4 0 0" size="0.06" />
<body name="r_elbow_flex_link" pos="0.4 0 0">
<geom name="ef" type="capsule" fromto="0 -0.02 0 0.0 0.02 0" size="0.06" />
<joint name="r_elbow_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-2.3213 0" damping="0.1" />
<body name="r_forearm_roll_link" pos="0 0 0">
<geom name="fr" type="capsule" fromto="-0.1 0 0 0.1 0 0" size="0.02" />
<joint name="r_forearm_roll_joint" type="hinge" limited="true" pos="0 0 0" axis="1 0 0" damping=".1" range="-1.5 1.5"/>
<body name="r_forearm_link" pos="0 0 0">
<geom name="fa" type="capsule" fromto="0 0 0 0.291 0 0" size="0.05" />
<body name="r_wrist_flex_link" pos="0.321 0 0">
<geom name="wf" type="capsule" fromto="0 -0.02 0 0 0.02 0" size="0.01" />
<joint name="r_wrist_flex_joint" type="hinge" pos="0 0 0" axis="0 1 0" range="-1.094 0" damping=".1" />
<body name="r_wrist_roll_link" pos="0 0 0">
<joint name="r_wrist_roll_joint" type="hinge" pos="0 0 0" limited="true" axis="1 0 0" damping="0.1" range="-1.5 1.5"/>
<body name="tips_arm" pos="0 0 0">
<geom name="tip_arml" type="sphere" pos="0.1 -0.1 0." size="0.01" />
<geom name="tip_armr" type="sphere" pos="0.1 0.1 0." size="0.01" />
</body>
<geom type="capsule" fromto="0 -0.1 0. 0.0 +0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 -0.1 0. 0.1 -0.1 0" size="0.02" contype="1" conaffinity="1" />
<geom type="capsule" fromto="0 +0.1 0. 0.1 +0.1 0." size="0.02" contype="1" conaffinity="1" />
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
</body>
<!--<body name="object" pos="0.55 -0.3 -0.275" >-->
<body name="object" pos="0.45 -0.05 -0.275" >
<!-- <geom rgba="1 1 1 0" type="sphere" size="0.05 0.05 0.05" density="0.001" conaffinity="0"/> -->
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.005" contype="1" conaffinity="0"/>
<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
<body name="goal" pos="0.45 -0.05 -0.3230">
<geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/>
<joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
</worldbody>
<actuator>
<motor joint="r_shoulder_pan_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_shoulder_lift_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_upper_arm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_elbow_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_forearm_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_flex_joint" ctrlrange="-2.0 2.0" ctrllimited="true" />
<motor joint="r_wrist_roll_joint" ctrlrange="-2.0 2.0" ctrllimited="true"/>
</actuator>
</mujoco>I trained pusher-v5rc5.xml
for 2 mil steps with SB3's SAC and we can see that the cylidrical object is light enough to be moved by the pusher arm, the objects's mass is 7.85398163e-06 kg, this should be good enough for release
next is 10 episodes per 10 polices trained
Pusher-v5rc5_SAC_run_0.mp4
Pusher-v5rc5_SAC_run_1.mp4
Pusher-v5rc5_SAC_run_2.mp4
Pusher-v5rc5_SAC_run_3.mp4
Pusher-v5rc5_SAC_run_4.mp4
Pusher-v5rc5_SAC_run_5.mp4
Pusher-v5rc5_SAC_run_6.mp4
Pusher-v5rc5_SAC_run_7.mp4
Pusher-v5rc5_SAC_run_8.mp4
Pusher-v5rc5_SAC_run_9.mp4
from gymnasium.
pseudo-rnd-thoughts
commented on September 13, 2024
@Kallinteris-Andreas wjat are the performance curves compared to pusher v4 and mujoco < 3
from gymnasium.
Ejafa
commented on September 13, 2024
is it okay? I saw somehow it's working
<!-- <body name="object" pos="0.55 -0.3 -0.275" > -->
<body name="object" pos="0.45 -0.05 -0.275" >
<geom rgba="1 1 1 0" type="sphere" `size="1` 0.05 0.05" density="0.00001" conaffinity="0"/>
<geom rgba="1 1 1 1" type="cylinder" size="0.05 0.05 0.05" density="0.00001" contype="1" conaffinity="0"/>
<joint name="obj_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="obj_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
<body name="goal" pos="0.45 -0.05 -0.3230">
<geom rgba="1 0 0 1" type="cylinder" size="0.08 0.001 0.1" density='0.00001' contype="0" conaffinity="0"/>
<joint name="goal_slidey" type="slide" pos="0 0 0" axis="0 1 0" range="-10.3213 10.3" damping="0.5"/>
<joint name="goal_slidex" type="slide" pos="0 0 0" axis="1 0 0" range="-10.3213 10.3" damping="0.5"/>
</body>
</worldbody>
from gymnasium.
Kallinteris-Andreas
commented on September 13, 2024
SB3 fork with support for gymnasium==1.0.0a1 was used
the environments are:
- Pusher-v5 the existing implementation of Pusher in
gymnasium==1.0.0a1withmujoco==2.3.7 - Pusher-v5-rc4 and -rc5 is the existing implementation of Pusher but with the new models
Analysis
the rc4 model is close enough to the original model, that it should be used as the for the Pusher-v5 release
@HernandezEduin are you also running tests?
from gymnasium.
pseudo-rnd-thoughts
commented on September 13, 2024
@Kallinteris-Andreas the results look good, I'm happy to merge this new model to replace the old v5 model
from gymnasium.
Related Issues (20)
- [To investigate] Check that the new autoreset order works correctly in vectorized jax envs HOT 3
- [Bug] info dictionary in `env.step()` does not include all claimed terms HOT 2
- [Bug Report] The description of the observation space of "half cheetah" is not correct on the documentation website. HOT 1
- [Question] Automated planning over simulators support HOT 4
- ERROR: Failed building wheel for box2d-py HOT 1
- [Bug Report] truncated is commented out HOT 1
- [Bug Report] shallow copy of info and observation after termination HOT 1
- [Bug Report] Error in Env Rendering Due to Current Dependency Installation HOT 1
- [Bug Report] Wrapper type hints seem to be specified incorrectly? HOT 1
- [New feature] Add reset options support for functional jax envs HOT 4
- [Bug Report] default_camera_config in MujocoEnv seems not working HOT 6
- [Bug Report] LunarLander continuous mode immediately raises exception due to incompatibility between numpy and box2d HOT 1
- [Bug Report] Custom environment documentation example gif missing HOT 2
- [Question] Wiki / Leaderboards? HOT 1
- [Bug Report] Possibly outdated docstring for TimeLimit max_episode_steps HOT 2
- [Bug Report] CarRacing car_dynamics.py TypeError HOT 1
- AssertionError: Values of the node_space should be instances of Box or Discrete, got <class 'int'> HOT 7
- [Bug Report] TransformObservation not backward compatible due to extra parameter HOT 3
- [Bug Report] No reward threshold for Walker2d-v4 HOT 5
- [Bug Report] gymnasium[all], pygame==2.1.3 not compatible with Python 3.12 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gymnasium.