nvidia-omniverse / isaacgymenvs Goto Github PK

View Code? Open in Web Editor NEW

1.7K 37.0 377.0 67.29 MB

Isaac Gym Reinforcement Learning Environments

License: Other

Batchfile 0.03% Python 99.93% CMake 0.02% Shell 0.01%

isaacgymenvs's People

Stargazers

Watchers

Forkers

haochihlin mcx lianhui1993 jrenaf sheim gavrielstate zhehui-huang denys88 munjuhyeok palanc athrva98 erwincoumans jonathan-kho kevinh42 dmarew kiyoshiiriemon prem-chand jaiveersingh1 taochenshh mhming sudhirpratapyadav vincentkslim creeperlin mihirk284 janisa9 namwoo andrewk1 algoskynet rpl-cs-ucl stevehao74 yangxiaojun1230 wzqwwq felipemartins96 sippeyxp boxixia jc-bao visakii-work stjordanis newera-001 y0ngw00 maniast collector-m skylark0924 colemakdvorak naokiyokoyama nikhilweee junjungoal alansunhr jaek709 tmats humarobot nyu-robot-learning elladyr neil0511 inarikami krishpop ritviksingh9 younghyopark fokx nickick-icrs ssheorey simondlevy cvoelcker khush3 onlyfuture robot-learning-library frank-dz btx0424 jacarvalho niklaska ankurhanda velythyl akanshumahajan xbpeng microsoft-fevieira vwxyzjn wsh95 zuxinrui kheele yixuanl milad-rakhsha fdoperezi samuel-tian dexairobotics jiahongwu1995 cnheider zikangxiong ar-mine niabubu aheadio taohuang13 weizx208 xibeisiber annadeichler marcin-projectx olivier-serris aod321 aiwintermuteai yin-tairan cloudbee7

isaacgymenvs's Issues

Success rate plotting

Hey,
thank you for the great work. I'm currently working with the Trifinger environment and wanted to ask if there is an easy way to add the success rate to the tensorboard logging? I checked already its in the environment step_info dictionary but I'm not sure how to add it to the logging. Also have you considered adding a wandb integration?
Thanks a lot!

AssertionError when changing number of environments

Hello, I have suddenly started getting an AssertionError, when changing the numEnvs in the RL examples. I thought my computer was messed up, so I reinstalled Ubuntu 20.04. My Isaac Gym is therefore running on a fresh system, however the error persists.

When I change the numEnvs parameter in the Cartpole.yaml from 512 to 256, i get the following error:

Error executing job with overrides: ['task=Cartpole']
Traceback (most recent call last):
  File "train.py", line 112, in launch_rlg_hydra
    'play': cfg.test,
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/torch_runner.py", line 139, in run
    self.run_train()
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/torch_runner.py", line 122, in run_train
    agent = self.algo_factory.create(self.algo_name, base_name='run', config=self.config)
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/common/object_factory.py", line 15, in create
    return builder(**kwargs)
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/torch_runner.py", line 23, in <lambda>
    self.algo_factory.register_builder('a2c_continuous', lambda **kwargs : a2c_continuous.A2CAgent(**kwargs))
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/algos_torch/a2c_continuous.py", line 18, in __init__
    a2c_common.ContinuousA2CBase.__init__(self, base_name, config)
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/common/a2c_common.py", line 973, in __init__
    A2CBase.__init__(self, base_name, config)
  File "/home/anton/anaconda3/envs/rlgpu/lib/python3.7/site-packages/rl_games/common/a2c_common.py", line 171, in __init__
    assert(self.batch_size % self.minibatch_size == 0)
AssertionError

Any help is much appreciated

Missing / Changed Environments

Hi,

It seems like several environments mentioned in the white papers are not included, e.g. Humanoid Character Animation (missing), Franka Cube Stacking (task changed to cabinet opening for Franka), also the terrain for the ANYmal Terrain version in the white papers is more complex than in the published code but I guess that can be changed easily. I am mostly bummed out about not including the Humanoid Character Animation environment with AMP. Any reason why these were not included/changed?

errors when using IsaacGymEnvs as a single module

After installed IsaacGym (preview3) and IsaacGymEnvs, I can run each demo in IsaacGymEnvs OK.
But there are some errors when I trying to import some function / class in IsaacGymEnvs from my project.

--
Issue 1:
After running the command:
$ cd IsaacGymEnvs ; pip install .

IsaacGymEnvs's code doesnot copy into python/**/site-package/, but only an egg file.

--
Issue 2:
By copying IsaacGymEnvs/isaacgymenvs to python/**/site-package/ manually, it cannot be import by third project outside.
for example:
from isaacgymenvs.tasks.base.vec_task import VecTask
raise an error:
.../site-packages/isaacgymenvs/tasks/init.py, line 30, in
from tasks.allegro_hand import AllegroHand
ModuleNotFoundError: No module named 'tasks'

Much thanks!

Is there a way to access the time step in the reward, for comparing with pre-recorded data

underwater environment

Hi, can we create an underwater environment for robots? Thank you!

Is there any way to access body linear and angular accelerations?

I can get positions and velocities withacquire_actor_root_state_tensor but accelerations are not provided, I checked examples like asset_info.py but still found nothing.

[Error] [carb.gym.plugin] *** Failed to load './/../assets/glb/ingenuity/chassis.glb'!

Importing the Ingenuity task/environment outside the IsaacGymEnvs/isaacgymenvs folder produces the following error:

(env_isaacgym4) toni@HP-ZBook-Studio-G8:~/Documents/examples/isaacgym$ python
Python 3.8.10 (default, Mar 15 2022, 12:22:08) 
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import isaacgym
Importing module 'gym_38' (/home/toni/isaacgym/isaacgym4/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/toni/isaacgym/isaacgym4/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
>>> import isaacgymenvs
>>> env = isaacgymenvs.make(seed=42, 
...                         task="Ingenuity", 
...                         num_envs=4096, 
...                         sim_device="cuda:0",
...                         rl_device="cuda:0",
...                         graphics_device_id=0,
...                         headless=False)
PyTorch version 1.12.0+cu116
Device count 1
/home/toni/isaacgym/isaacgym4/python/isaacgym/_bindings/src/gymtorch
Using /home/toni/.cache/torch_extensions/py38_cu116 as PyTorch extensions root...
Emitting ninja build file /home/toni/.cache/torch_extensions/py38_cu116/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
/home/toni/isaacgym/isaacgym4/python/isaacgym/torch_utils.py:135: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  def get_axis_params(value, axis_idx, x_value=0., dtype=np.float, n_dims=3):
2022-07-11 17:25:28,797 - INFO - logger - logger initialized
<unknown>:3: DeprecationWarning: invalid escape sequence \*
Error: FBX library failed to load - importing FBX data will not succeed. Message: No module named 'fbx'
FBX tools must be installed from https://help.autodesk.com/view/FBX/2020/ENU/?guid=FBX_Developer_Help_scripting_with_python_fbx_installing_python_fbx_html
/home/toni/isaacgym/isaacgym4/IsaacGymEnvs/isaacgymenvs/__init__.py:35: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  with initialize(config_path="./cfg"):
/home/toni/Documents/envs/env_isaacgym4/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
  warnings.warn(msg, UserWarning)
/home/toni/Documents/envs/env_isaacgym4/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:415: UserWarning: In config: Invalid overriding of hydra/job_logging:
Default list overrides requires 'override' keyword.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/defaults_list_override for more information.

  deprecation_warning(msg)
/home/toni/Documents/envs/env_isaacgym4/lib/python3.8/site-packages/gym/spaces/box.py:112: UserWarning: WARN: Box bound precision lowered by casting to float32
  logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
[Warning] [carb.gym.plugin] useGpu is set, forcing single scene (0 subscenes)
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

[Error] [carb.gym.plugin] *** Failed to load './/../assets/glb/ingenuity/chassis.glb'!

Segmentation fault (core dumped)

Problem: The path to the assets is relative only to the isaacgymenvs python package.

Steps to reproduce:

Open python (interactive terminal) in any folder except IsaacGymEnvs/isaacgymenvs
Execute the following code

import isaacgym
import isaacgymenvs
env = isaacgymenvs.make(seed=42, 
                      task="Ingenuity", 
                      num_envs=4096, 
                      sim_device="cuda:0",
                      rl_device="cuda:0",
                      graphics_device_id=0,
                      headless=False)

Creating a T-Pose .npy file with Poselib

Hi,

I’m failing to create a T-Pose .npy. I want to retarget the SFU Wushu motion to the AMP skeleton. As I do not have any MJCF file from them but only FBX, I tried to extract the T-Pose from the FBX and saved it as another FBX. However, when I run fbx_importer on it it creates a SkeletonMotion but the retargeting expects a SkeletonState for the T-Pose. In the example a SkeletonState is only created from a MJCF file however.

Best,
Noshaba

Making constraints or joints between two rigid bodies

I would like to know how to make constraints or joints between two rigid bodies. To be specific, I would like to simulate a suction-like gripper.

Using an attractor seems to work, but it does not support GPU pipelines. I also tried explicitly applying impedance force on the bodies to implement such a function, but it does not seem stable.

Observation Buffer Size in Humanoid

Hi,

I think the comment in line 407 in humanoid.py needs to be changed to

obs_buf shapes: 1, 3, 3, 1, 1, 1, 1, 1, num_dofs (21), num_dofs (21), num_dofs (21), 12, num_acts (21)

Wrong(?) next observation after reset

For all tasks in the example files, the post physics step looks like the following.

env_ids = self.reset_buf.nonzero(as_tuple=False).squeeze(-1)
if len(env_ids) > 0:
    self.reset_idx(env_ids)

self.compute_observations()
self.compute_reward()

Say we now reach maximum episode and env 1 needs to be reset. The pre physics step was simulated based on the pre-reset observation, however the post physics step after reset will give a next reset observation and reward which is not consistent?

It seems that only the first step right after the reset has this inconsistency issue, so fixing this does not has a significant performance improvement?

Or did I miss something here?

Segmentation fault (core dumped)

When I ran the example file python train.py task=Ant headless=True, I got an error as Segmentation fault (core dumped). The full output is as follows:

Importing module 'gym_38' (/home/liuchi/zhaoyinuo/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/liuchi/zhaoyinuo/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
PyTorch version 1.11.0+cu102
Device count 8
/home/liuchi/zhaoyinuo/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/liuchi/.cache/torch_extensions/py38_cu102 as PyTorch extensions root...
Emitting ninja build file /home/liuchi/.cache/torch_extensions/py38_cu102/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
/home/liuchi/zhaoyinuo/isaacgym/python/isaacgym/torch_utils.py:135: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this
 will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  def get_axis_params(value, axis_idx, x_value=0., dtype=np.float, n_dims=3):
2022-06-27 10:44:06,084 - INFO - logger - logger initialized
<unknown>:3: DeprecationWarning: invalid escape sequence \*
Error: FBX library failed to load - importing FBX data will not succeed. Message: No module named 'fbx'
FBX tools must be installed from https://help.autodesk.com/view/FBX/2020/ENU/?guid=FBX_Developer_Help_scripting_with_python_fbx_installing_python_fbx_html
/home/liuchi/anaconda3/envs/hand/lib/python3.8/site-packages/torch/utils/tensorboard/__init__.py:4: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if not hasattr(tensorboard, '__version__') or LooseVersion(tensorboard.__version__) < LooseVersion('1.15'):
train.py:66: UserWarning:
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name="config", config_path="./cfg")
/home/liuchi/anaconda3/envs/hand/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.
1/default_composition_order for more information
  warnings.warn(msg, UserWarning)
/home/liuchi/anaconda3/envs/hand/lib/python3.8/site-packages/hydra/_internal/defaults_list.py:415: UserWarning: In config: Invalid overriding of hydra/job_logging:
Default list overrides requires 'override' keyword.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/defaults_list_override for more information.

  deprecation_warning(msg)
/home/liuchi/anaconda3/envs/hand/lib/python3.8/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
  ret = run_job(
task:
    name: Ant
    physics_engine: physx
    env:
        numEnvs: 4096
        envSpacing: 5
        episodeLength: 1000
        enableDebugVis: False
        clipActions: 1.0
        powerScale: 1.0
        controlFrequencyInv: 1
        headingWeight: 0.5
        upWeight: 0.1
        actionsCost: 0.005
        energyCost: 0.05
        dofVelocityScale: 0.2
        contactForceScale: 0.1
        jointsAtLimitCost: 0.1
        deathCost: -2.0
        terminationHeight: 0.31
        plane:
            staticFriction: 1.0
            dynamicFriction: 1.0
            restitution: 0.0
        asset:
            assetFileName: mjcf/nv_ant.xml
        enableCameraSensors: False
    sim:
        dt: 0.0166
        substeps: 2
        up_axis: z
        use_gpu_pipeline: True
        gravity: [0.0, 0.0, -9.81]
        physx:
            num_threads: 4
            solver_type: 1
            use_gpu: True
            num_position_iterations: 4
            num_velocity_iterations: 0
            contact_offset: 0.02
            rest_offset: 0.0
            bounce_threshold_velocity: 0.2
            max_depenetration_velocity: 10.0
            default_buffer_size_multiplier: 5.0
            max_gpu_contact_pairs: 8388608
            num_subscenes: 4
            contact_collection: 0
    task:
        randomize: False
        randomization_params:
            frequency: 600
            observations:
                range: [0, 0.002]
                operation: additive
                distribution: gaussian
            actions:
                range: [0.0, 0.02]
                operation: additive
                distribution: gaussian
            actor_params:
                ant:
                    color: True
                    rigid_body_properties:
                        mass:
                            range: [0.5, 1.5]
                            operation: scaling
                            distribution: uniform
                            setup_only: True
                    dof_properties:
                        damping:
                            range: [0.5, 1.5]
                            operation: scaling
                            distribution: uniform
                        stiffness:
                            range: [0.5, 1.5]
                            operation: scaling
                            distribution: uniform
                        lower:
                            range: [0, 0.01]
                            operation: additive
                            distribution: gaussian
                        upper:
                            range: [0, 0.01]
                            operation: additive
                            distribution: gaussian
train:
    params:
        seed: 42
        algo:
            name: a2c_continuous
        model:
            name: continuous_a2c_logstd
        network:
            name: actor_critic
            separate: False
            space:
                continuous:
                    mu_activation: None
                    sigma_activation: None
                    mu_init:
                        name: default
                    sigma_init:
                        name: const_initializer
                        val: 0
                    fixed_sigma: True
            mlp:
                units: [256, 128, 64]
                activation: elu
                d2rl: False
                initializer:
                    name: default
                regularizer:
                    name: None
        load_checkpoint: False
        load_path:
        config:
            name: Ant
            full_experiment_name: Ant
            env_name: rlgpu
            multi_gpu: False
            ppo: True
            mixed_precision: True
            normalize_input: True
            normalize_value: True
            value_bootstrap: True
            num_actors: 4096
            reward_shaper:
                scale_value: 0.01
            normalize_advantage: True
            gamma: 0.99
            tau: 0.95
            learning_rate: 0.0003
            lr_schedule: adaptive
            schedule_type: legacy
            kl_threshold: 0.008
            score_to_win: 20000
            max_epochs: 500
            save_best_after: 200
            save_frequency: 50
            grad_norm: 1.0
            entropy_coef: 0.0
            truncate_grads: False
            e_clip: 0.2
            horizon_length: 16
            minibatch_size: 32768
            mini_epochs: 4
            critic_coef: 2
            clip_value: True
            seq_len: 4
            bounds_loss_coef: 0.0001
task_name: Ant
experiment:
num_envs:
seed: 42
torch_deterministic: False
max_iterations:
physics_engine: physx
pipeline: gpu
sim_device: cuda:0
rl_device: cuda:0
graphics_device_id: 0
num_threads: 4
solver_type: 1
num_subscenes: 4
test: False
checkpoint:
multi_gpu: False
headless: True
Setting seed: 42
self.seed = 42
Exact experiment name requested from command line: Ant
num_observations:  60 ; num_states: 0
/home/liuchi/anaconda3/envs/hand/lib/python3.8/site-packages/gym/spaces/box.py:84: UserWarning: WARN: Box bound precision lowered by casting to float32
  logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
[Warning] [carb.gym.plugin] useGpu is set, forcing single scene (0 subscenes)
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Segmentation fault (core dumped)

I also tried to run the example in isaacgym/python/examples/joint_monkey.py, but still got an error as Segmentation fault (core dumped). I used to run successfully in this server, but now, after upgrade the ubuntu-drivers, I always got a Segmentation fault error. Even I changed the ubuntu-drivers back, the error still exists.

Here is the output of nvidia-smi

(hand) liuchi@omnisky:~/zhaoyinuo/IsaacGymEnvs/isaacgymenvs$ nvidia-smi
Mon Jun 27 10:54:18 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.129.06   Driver Version: 470.129.06   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA TITAN RTX    Off  | 00000000:1A:00.0 Off |                  N/A |
| 41%   30C    P8     3W / 280W |    202MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA TITAN RTX    Off  | 00000000:1B:00.0 Off |                  N/A |
| 41%   29C    P8     4W / 280W |    180MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA TITAN RTX    Off  | 00000000:3D:00.0 Off |                  N/A |
| 41%   27C    P8     8W / 280W |    180MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   3  NVIDIA GeForce ...  Off  | 00000000:3E:00.0 Off |                  N/A |
| 37%   26C    P8    25W / 350W |    272MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   4  NVIDIA TITAN RTX    Off  | 00000000:88:00.0 Off |                  N/A |
| 41%   28C    P8     6W / 280W |    180MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   5  NVIDIA TITAN RTX    Off  | 00000000:89:00.0 Off |                  N/A |
| 41%   28C    P8     8W / 280W |    180MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   6  NVIDIA TITAN RTX    Off  | 00000000:B1:00.0 Off |                  N/A |
| 41%   30C    P8    13W / 280W |    180MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   7  NVIDIA TITAN RTX    Off  | 00000000:B2:00.0 Off |                  N/A |
| 41%   28C    P8    19W / 280W |    180MiB / 24220MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

My platform is ubuntu18.04.

Trained Trifinger model does not learn to lift the cube up

After running the train script for the Trifinger task, the learned model does not learn how to pick up the cube from the ground, rather it flips the cube over until it is close to the goal position with one finger only, then balances the cube so that there is some overlap but it does not learn as reported in the paper

ONNX Export Checkpoints

It would be nice if trained Isaac Gym models checkpointed to ONNX as well (for additional portability to game engines).

Here, I'll document the steps in my adventure so far in getting .onnx models out of the system:

Update the PyTorch version to 1.12. I did this by updating isaacgym's Dockerfile's base image to nvcr.io/nvidia/pytorch:22.04-py3. Thankfully, it just works. This is necessary because 1.10's ONNX Exporter can't handle the random normal-distribution operation. If ONNX Export were to be made standard, this would need to be propagated to the proper Isaac Gym preview: https://developer.nvidia.com/isaac-gym
Augment the run.sh command with another volume to allow for retrieval of the trained models (adding -v /home/gymuser/IsaacGymEnvs:/home/gymuser/IsaacGymEnvs is sufficient; we're going to pull this repo into that folder in the next step).
After starting the container, run:

git clone https://github.com/NVIDIA-Omniverse/IsaacGymEnvs.git ~/IsaacGymEnvs
pip install -q -e ~/IsaacGymEnvs

At this point, we're going to want to make some changes to the isaacgymenvs/learning/common_agent.py (to write .onnx models with each checkpoint). I did this by attaching a VS Code instance with the Docker Extension, but nano works as well. At the top under from torch import optim, add:

from torch.onnx import utils as onnx_utils

Underneath where it says self.save(self.model_output_file + "_" + str(epoch_num)), add:

                        # Grab some dummy inputs for the onnx.export function
                        input_dict = self.dataset[len(self.dataset)-1]
                        input_dict['is_train'] = False
                        input_dict['prev_actions'] = input_dict['actions']
                        # Reduce the Batch Size down to 1 for Inference
                        for key in input_dict:
                            if key != "is_train": # Ignore the bool
                                input_dict[key] = input_dict[key][0:1]
                                print("Name: ", key, ", Shape: ", input_dict[key].shape)
                        # If the conversion is about to fail, print the unconvertible ops:
                        torch_script_graph, unconvertible_ops = onnx_utils.unconvertible_ops(
                            self.model, input_dict, opset_version = 11)
                        if len(unconvertible_ops) > 0:
                            print("Operations Incompatible with ONNX Export: ", unconvertible_ops)
                        else:
                            torch.onnx.export(self.model, input_dict, self.model_output_file + "_" + str(epoch_num)+".onnx",
                                            export_params       = True,  # Whether to store the trained parameter weights inside the model file
                                            opset_version       = 11,    # The ONNX version to export the model to
                                            do_constant_folding = True,  # Whether to execute constant folding (hardcoding of constant nodes) for optimization
                                            input_names         = ['is_train', 'prev_actions', 'amp_obs_demo', 'amp_obs_replay', 'amp_obs', 'obs', 'old_values', 'old_logp_actions'], # Not sure these are correct
                                            output_names        = ['advantages', 'returns', 'actions', 'mu', 'sigma' ], # Likewise, I'm not sure if these are correct...
                                            verbose             = False) # If True, print a model summary to the console

It would be nice if one of the authors could check my work here; I'm not sure if I have the names of the input and output tensors correct... @gavrielstate

After training for >50 epochs (via python train.py task=HumanoidAMP or somesuch), you should be seeing .onnx checkpoints dumped alongside your PyTorch .pth checkpoints. I've attached an example HumanoidAMP checkpoint ( HumanoidAMP_3455_ONNX.zip ), which can be inspected in https://netron.app/ (and hopefully run in Unity's Barracuda Evaluator; I haven't tested it yet).

There's a strong chance I'm not properly accounting for inputs, persistent state, or the AMP actor critic properly... but I'm hoping my explorations here help lay the groundwork for more comprehensive ONNX support and portability across the Isaac Gym Ecosystem.

Thank you for your consideration.

The AMP method in IsaacGym is not the same as the AMP method in the paper

I see that the GAN loss function used in your AMP algorithm example is not the same as the loss function mentioned in the AMP paper. You are using something like "-E[log(D(s,s'))]-E[log(1-D(s,s'))]" as the formula (5) mentioned in the AMP paper, But they used formula (6) from LSGAN.
The reward function and gradient penalty function are also different from those mentioned in the AMP paper.
I'm wondering why you guys have a different approach than AMP? Or am I looking at something wrong?

Will DeepMimic/AMP in Isaac Gym be open sourced?

Is there any function for ray detection or collision detection?

Since I want to simulate a radar-like function, the ray detection function is required. I know that there is ray detection in PhysX. Through ray detection, you can get the information of the hit object, and there is also collision detection. When an object collides, it will return a series of collision information, etc., but I see that there is no relevant content in isaacgym. Are there functions related to ray detection and collision detection?

how can i modify the number of envs in the Anymal demo?

Anymal demo goes well when i don'tmodify the num of envs in the Anymal.yaml file, but when i modify it the simulition can't run and there is a AssertionError in a2c_common.py: assert(self.batch_size%self.minibatch_size == 0)

thanks!!!

How to export the sim env to usda?

Hi~ Thanks for the great work Isaacgym! I am playing with the demo Ant.
I just curious how can we export the scene we build in isaacgym?
I tried to export the sim using:

        usda_exporter = self.gym.create_usd_exporter()
        self.gym.set_usd_export_root(usda_exporter, "./export")
        self.gym.export_usd_sim(usda_exporter,  self.sim, "scene", "export")

but got the ground plane only:

Also see my post in the forum:
https://forums.developer.nvidia.com/t/how-to-export-the-sim-env-to-usda/218708

Requiring simpler API for designating physical parameters

Dear Developers in NVIDIA:
Thank you for your impressive work on this repository!
I'm launching a transfer learning research program in robotics. In the program, I should train policies in environments with different physical parameters and then test them in a certain one with specified parameters. However, I found it not so straightforward to designate the parameter vector using the current IsaacGymEnvs codebase --- I could only control the range of randomization of each component. Further, some parameters (e.g., the friction coefficients of all the 7 DOFs) share the same randomization range, so never could I specify an interval of length 0 to emulate the specification process.
I would appreciate it if you could update the repository with some finer-grained domain randomization APIs where I could specify the parameters directly!

how can i train it on remote system.

Hi, thx for the great job on IsaacGymEnvs!
I don’t own a local desktop machine, I rely on remote cluster for GPUs. I have headless docker + ssh./ conda + ssh. ,(i tried VNC, but it seems not good on isaac gym)
All the python/examples show segfault, RL env report the same segfault core dump error. I looked up a lot of methods but nothing worked. How can i train it just on terminal?

How is the shadowhand fixed the in the air?

Hi,

I am trying to do a personal project regarding manipulators.
However, when I load my mjcf asset, it always drops to the ground.
I noticed that in the ShadowHand example, the base of the hand is always floating in the air.

I am new to this enviroment and is it possible to point out which part of the code should I look at in order to have the same effect as shadow hand?

Best,

reference to world coordinates and acquire_actor_root_state_tensor

Does acquire_actor_root_state_tensor output world or reference coordinates?
How could I convert output coordinates to world coordinates

Pointers to create a MA environment

Hi all,
I'm trying to recreate the RoboSumo environment https://github.com/openai/robosumo . Could someone give me an overview / code pointers to create this? It would essentially be two Ants whose actions affect each other, along with a simple mat texture on the bottom to indicate the "ring".

$python train.py task=Cartpole task.sim.gravity=[100,100,9.81] doesn't work

I know this is really useless thing but I wanna know how to control simulation conditions.
I tried
isaacgym/IsaacGymEnvs/isaacgymenvs/cfg/config.yaml
Line 47: Ant -> ${task.name}

So, the terminal it said
'''bash
sim:
dt: 0.0166
substeps: 2
up_axis: z
use_gpu_pipeline: True
gravity: [100, 100, 9.8]
physx:
num_threads: 4
solver_type: 1
use_gpu: True
num_position_iterations: 4
num_velocity_iterations: 0
contact_offset: 0.02
rest_offset: 0.001
bounce_threshold_velocity: 0.2
max_depenetration_velocity: 100.0
default_buffer_size_multiplier: 2.0
max_gpu_contact_pairs: 1048576
num_subscenes: 4
contact_collection: 0
'''
I can see it works properly but in simulations'
sim category

it still show 0,0,-9.81

How can I control it?

internal error : PhysX Internal CUDA error. Simulation can not continue! FIXED

hi, i faced a problem with exact description like this
"/buildAgent/work/45f70df4210b2e3e/source/physx/src/NpScene.cpp (2869) : internal error : PhysX Internal CUDA error. Simulation can not continue!
/buildAgent/work/45f70df4210b2e3e/source/physx/src/NpScene.cpp (3175) : invalid operation : PxScene::fetchResults: fetchResults() called illegally! It must be called after advance() or simulate()
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: ../../../source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 3362"
and ı fix problem with edit my np.full{} velocity parameter to gived to dof
your solve si couldnt be same like me but the only thing ı know that error could source from anything

Virtual environment installations can't run joint_monkey.py

On a machine without GPU:

lsb_release -a  # Ubuntu 20.04.4 LTS
python3 --version # Python 3.8.10
python3 -m venv ~/.virtualenvs/isaacgym
source ~/.virtualenvs/isaacgym/bin/activate
cd python
pip install -e .
cd examples
python joint_monkey.py

This gives:

*** Warning: failed to preload CUDA lib
*** Warning: failed to preload PhysX libs
Importing module 'gym_38' (/home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
[Error] [carb] [Plugin: libcarb.gym.plugin.so] Could not load the dynamic library from /home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/libcarb.gym.plugin.so. Error: libcuda.so.1: cannot open shared object file: No such file or directory
[Warning] [carb] Potential plugin preload failed: /home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/libcarb.gym.plugin.so
Setting GYM_USD_PLUG_INFO_PATH to /home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
[Error] [carb] Failed to acquire interface: [carb::gym::Gym v0.1], by client: carb.gym.python.gym_38 (plugin name: (null))
Traceback (most recent call last):
  File "joint_monkey.py", line 59, in <module>
    gym = gymapi.acquire_gym()
RuntimeError: Failed to acquire interface: carb::gym::Gym (pluginName: nullptr)

I then go to a Ubuntu 18.04.5 LTS machine with CUDA 11.1 and an RTX 2080 Ti with the 455.45.01 driver (per nvidia-smi).

sudo apt install python3.8 python3.8-dev  # from http://ppa.launchpad.net/deadsnakes/ppa/ubuntu bionic/main amd64 Packages
mkvirtualenv --python=/usr/bin/python3.8 isaacgym  # 3.8.13
workon isaacgym
cd python
pip install -e .
cd examples
python joint_monkey.py

and get

Importing module 'gym_38' (/home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_38.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
WARNING: Forcing CPU pipeline.
Not connected to PVD
/buildAgent/work/45f70df4210b2e3e/source/cudamanager/src/CudaContextManager.cpp (635) : warning : CUDA driver version is 11010, expected at least 11040.

/buildAgent/work/45f70df4210b2e3e/source/cudamanager/src/CudaContextManager.cpp (370) : info : NVIDIA Release 411 graphics driver and above is required for GPU acceleration.

[Warning] [carb.gym.plugin] Failed to create a valid PhysX CUDA Context Manager. Falling back to CPU.
Physics Engine: PhysX
Physics Device: cpu
GPU Pipeline: disabled
Segmentation fault (core dumped)

I also try with Miniconda on the 18.04 machine:

conda activate base
./create_conda_env_rlgpu.sh
conda activate rlgpu
cd python/examples/
python joint_monkey.py

I get

Importing module 'gym_37' (/home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_37.so)
Traceback (most recent call last):
  File "joint_monkey.py", line 19, in <module>
    from isaacgym import gymapi, gymutil
  File "/home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/__init__.py", line 5, in <module>
    from isaacgym import gymapi
  File "/home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/gymapi.py", line 104, in <module>
    _import_active_version()
  File "/home/tsbertalan/Dropbox/Projects/ODEs for RL/isaacgym/python/isaacgym/gymapi.py", line 63, in _import_active_version
    module = importlib.import_module(package_path)
  File "/home/tsbertalan/miniconda3/envs/rlgpu/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory

I am able to run the Omniverse installer (on the first machine), do its 2 or 3GB GUI installer for "Isaac SIm", and then launch that GUI. But this seems like a totally different thing from the gym environments here, and the terminal it shows displays a lot of errors, though a GUI appears and is clickable.

Multiple anymal robots appearing in the same coordiante - AnimalTerrain

Hey I was trying out the example Animal Terrain .
I assume each Anymal was suppose to appear in a different location?
but this is what happend:

In another Question - Is multiple gpu supported in ths issac Gym?

RuntimeError: CUDA error: an illegal memory access was encountered

I keep getting this at random time periods during my training. The specific error:

/buildAgent/work/45f70df4210b2e3e/source/gpunarrowphase/src/PxgNarrowphaseCore.cpp (9908) : internal error : GPU compressContactStage1 fail to launch kernel stage 1!!

/buildAgent/work/45f70df4210b2e3e/source/gpunarrowphase/src/PxgNarrowphaseCore.cpp (9945) : internal error : GPU compressContactStage2 fail to launch kernel stage 1!!

Is it possible to use Isaac Gym without simulating physics?

Hi all, thank you for the awesome work you've done in creating Isaac Gym!

I'm wondering if it's possible to use Isaac Gym without making use of any physics simulation.

More specifically, it seems that dof_state_tensor and actor_root_state_tensor all require physical actors to be simulated with gravity, ground planes, etc... but what if we'd like to train with Isaac Gym on a non-physically-based RL task? For example, the CarRacing environment from OpenAI or some other task in healthcare, transportation, finance, etc.?

I would think it's possible to do this with torch.jit functions by:

initializing random state tensors for each environment,
applying actions and stepping the environment in the pre_physics_step(),
computing rewards and observations in the post_physics_step(),
repeating all this as desired.

However, I've gone through a lot of the documentation and haven't found any way to initialize states without specifying some sort of physical actor. Specifying actors wouldn't make sense if we were trying to train on CarRacing, for example.

The fully accelerated RL paradigm in Isaac Gym is extremely powerful and I feel it would be huuugely beneficial to the RL community to be able to train any task fully on the GPU, even if non-physical.

Any thoughts or advice?

Thank you,
Momin

Getting image observations from environment

Hello,

If I am running on a headless server, is there anyway for me to get image observations from the environment so that I can generate/save a video of the agent's performance? Something like gym's env.render("rgb_array") that I can use when I have set headless=True.

Thanks!

OmniGym

Hi, thank you for the great work! The latest GTC shows the OmniGym. When can we try it?

Cartpole: Gym cuda error: no kernel image is available for execution on the device

When I run example python train.py task=Cartpole I have got error:

[Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 991
[Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 1010
[Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 926
[Error] [carb.gym.plugin] Failed to fill rigid body state tensor
[Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 991
[Error] [carb.gym.plugin] Gym cuda error: no kernel image is available for execution on the device: ../../../source/plugins/carb/gym/impl/Gym/GymPhysXCuda.cu: 1010
fps step: 18839.4 fps step and policy inference: 17392.6  fps total: 15201.0
=> saving checkpoint 'runs/Cartpole/nn/last_Cartpoleep101rew[497.33].pth'
MAX EPOCHS NUM!

Cannot clone repo: "This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access."

I tried cloning the repo, but it seems some objects are stored in the LFS and the bandwidth is all used up.
Whenever I try to clone the repo, first, I get

Cloning into 'IsaacGymEnvs'...
remote: Enumerating objects: 568, done.
remote: Counting objects: 100% (568/568), done.
remote: Compressing objects: 100% (450/450), done.
remote: Total 568 (delta 114), reused 568 (delta 114), pack-reused 0
Receiving objects: 100% (568/568), 25.76 MiB | 9.37 MiB/s, done.
Resolving deltas: 100% (114/114), done.
Downloading assets/trifinger/objects/meshes/ball.obj (5.6 KB)
Error downloading object: assets/trifinger/objects/meshes/ball.obj (88c1693): Smudge error: Error downloading assets/trifinger/objects/meshes/ball.obj (88c16939f216d501a06685b476edf01e8e41f01c4124882549186534c6cee39d): batch response: Post https://github.com/JohannLange/IsaacGymEnvs.git/info/lfs/objects/batch: x509: certificate signed by unknown authority

Errors logged to /home/johannlange/tmp/IsaacGymEnvs/.git/lfs/logs/20211105T151111.3103445.log
Use `git lfs logs last` to view the log.
error: external filter 'git-lfs filter-process' failed
fatal: assets/trifinger/objects/meshes/ball.obj: smudge filter lfs failed
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry with 'git restore --source=HEAD :/'

Afterwards, when I try to manually fetch the LFS objects, I get

fetch: Fetching reference refs/heads/main
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
error: failed to fetch some objects from 'https://github.com/NVIDIA-Omniverse/IsaacGymEnvs.git/info/lfs'

I can reproduce the issue via:

git clone https://github.com/NVIDIA-Omniverse/IsaacGymEnvs.git
cd IsaacGymEnvs/
git lfs fetch

Is there any solution to this?

PS: in case you have some certificate issues, run git config http.sslVerify false before running git lfs fetch.

A quickstart guide for RL researchers

Is there an easy way to use IsaacGym environments to test research code? I've been struggling to do the equivalent of this with IsaacGym for hours now despite sifting through tons of documentation. A quick-start guide like below would greatly help researchers who may not necessarily want to commit to rl_games or use the train.py script, but quickly integrate their own code & torch models for benchmarking with little-to-no overhead.

Gym.prepare_sim returns False when num of bodys in single assert more than 256

Hi, thank you for the great job. I am trying to create an in-hand manipulation RL env on Isaac gym. A good example is shown in IsaacGymEnvs/allegro_hand.py at main · NVIDIA-Omniverse/IsaacGymEnvs · GitHub and this example runs well on my computer.

However, the real robot I used is more complicated than the original allegro hand and contains more than 256 bodies ( about 700 bodies). When I replaced the allegro hand path with my own robot hand path, gym.prepare_sim() returns False. After that, I can’t get any state tensor like rigid_body_state by

_rb_states = gym.acquire_rigid_body_state_tensor(sim)
rb_states = gymtorch.wrap_tensor(_rb_states)

The error details are shown below after gym.prepare_sim()
[Error] [carb.gym.plugin] Gym cuda error: an illegal memory access was encountered: …/…/…/source/plugins/carb/gym/impl/Gym/GymPhysX.cpp: 1370

If I try to get the rb_states, a CUDA error will show as below:
RuntimeError: CUDA error: an illegal memory access was encountered

After test, the env runs well when I reduce the number of robot bodies to 256 and raise error when 257. I’m guessing that maybe Isaac gym set a maximum number 256 of bodies for each assert. Could I change this or did I miss something? Any suggestions will be very helpful. Thank you!

Here is some extra information:
The conda env is created by official documents.
torch.version == “1.8.1”
isaacgym.version ==‘1.0.preview3’
python == 3.7
Nvidia Driver: 470.103.01
CUDA Version: 11.4

Failed to create SDF triangle mesh

Hi,

I'm trying to use factory environments for training. Specifically, with python train.py task=FactoryTaskNutBoltPick. I receive the error [Error] [carb.gym.plugin] Failed to create SDF triangle mesh. And the full log of error report is as following:

Importing module 'gym_37' (/home/zihan/software/isaacgym/python/isaacgym/_bindings/linux-x86_64/gym_37.so)
Setting GYM_USD_PLUG_INFO_PATH to /home/zihan/software/isaacgym/python/isaacgym/_bindings/linux-x86_64/usd/plugInfo.json
train.py:49: UserWarning: 
The version_base parameter is not specified.
Please specify a compatability version level, or None.
Will assume defaults for version 1.1
  @hydra.main(config_name="config", config_path="./cfg")
/home/zihan/anaconda3/envs/x/lib/python3.7/site-packages/hydra/_internal/defaults_list.py:251: UserWarning: In 'config': Defaults list is missing `_self_`. See https://hydra.cc/docs/upgrades/1.0_to_1.1/default_composition_order for more information
  warnings.warn(msg, UserWarning)
/home/zihan/anaconda3/envs/x/lib/python3.7/site-packages/hydra/_internal/defaults_list.py:415: UserWarning: In config: Invalid overriding of hydra/job_logging:
Default list overrides requires 'override' keyword.
See https://hydra.cc/docs/next/upgrades/1.0_to_1.1/defaults_list_override for more information.

  deprecation_warning(msg)
/home/zihan/anaconda3/envs/x/lib/python3.7/site-packages/hydra/_internal/hydra.py:127: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default.
See https://hydra.cc/docs/next/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information.
  configure_logging=with_log_configuration,
PyTorch version 1.11.0.dev20220110
Device count 8
/home/zihan/software/isaacgym/python/isaacgym/_bindings/src/gymtorch
Using /home/zihan/.cache/torch_extensions/py37_cu113 as PyTorch extensions root...
Emitting ninja build file /home/zihan/.cache/torch_extensions/py37_cu113/gymtorch/build.ninja...
Building extension module gymtorch...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
ninja: no work to do.
Loading extension module gymtorch...
/home/zihan/software/isaacgym/python/isaacgym/torch_utils.py:135: DeprecationWarning: `np.float` is a deprecated alias for the builtin `float`. To silence this warning, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  def get_axis_params(value, axis_idx, x_value=0., dtype=np.float, n_dims=3):
2022-07-05 14:30:06,887 - INFO - logger - logger initialized
<unknown>:6: DeprecationWarning: invalid escape sequence \*
Error: FBX library failed to load - importing FBX data will not succeed. Message: No module named 'fbx'
FBX tools must be installed from https://help.autodesk.com/view/FBX/2020/ENU/?guid=FBX_Developer_Help_scripting_with_python_fbx_installing_python_fbx_html
task: 
    name: FactoryTaskNutBoltPick
    physics_engine: physx
    sim: 
        use_gpu_pipeline: True
        up_axis: z
        dt: 0.016667
        gravity: [0.0, 0.0, -9.81]
        disable_gravity: False
    env: 
        numEnvs: 128
        numObservations: 20
        numActions: 12
        close_and_lift: True
        num_gripper_move_sim_steps: 20
        num_gripper_close_sim_steps: 25
        num_gripper_lift_sim_steps: 25
    randomize: 
        franka_arm_initial_dof_pos: [0.3413, -0.8011, -0.067, -1.8299, 0.0266, 1.0185, 1.0927]
        fingertip_midpoint_pos_initial: [0.0, -0.2, 0.2]
        fingertip_midpoint_pos_noise: [0.2, 0.2, 0.1]
        fingertip_midpoint_rot_initial: [3.1416, 0, 3.1416]
        fingertip_midpoint_rot_noise: [0.3, 0.3, 1]
        nut_pos_xy_initial: [0.0, -0.3]
        nut_pos_xy_initial_noise: [0.1, 0.1]
        bolt_pos_xy_initial: [0.0, 0.0]
        bolt_pos_xy_noise: [0.1, 0.1]
    rl: 
        pos_action_scale: [0.1, 0.1, 0.1]
        rot_action_scale: [0.1, 0.1, 0.1]
        force_action_scale: [1.0, 1.0, 1.0]
        torque_action_scale: [1.0, 1.0, 1.0]
        clamp_rot: True
        clamp_rot_thresh: 1e-06
        num_keypoints: 4
        keypoint_scale: 0.5
        keypoint_reward_scale: 1.0
        action_penalty_scale: 0.0
        max_episode_length: 100
        success_bonus: 0.0
    ctrl: 
        ctrl_type: joint_space_id
        all: 
            jacobian_type: geometric
            gripper_prop_gains: [50, 50]
            gripper_deriv_gains: [2, 2]
        gym_default: 
            ik_method: dls
            joint_prop_gains: [40, 40, 40, 40, 40, 40, 40]
            joint_deriv_gains: [8, 8, 8, 8, 8, 8, 8]
            gripper_prop_gains: [500, 500]
            gripper_deriv_gains: [20, 20]
        joint_space_ik: 
            ik_method: dls
            joint_prop_gains: [1, 1, 1, 1, 1, 1, 1]
            joint_deriv_gains: [0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
        joint_space_id: 
            ik_method: dls
            joint_prop_gains: [40, 40, 40, 40, 40, 40, 40]
            joint_deriv_gains: [8, 8, 8, 8, 8, 8, 8]
        task_space_impedance: 
            motion_ctrl_axes: [1, 1, 1, 1, 1, 1]
            task_prop_gains: [40, 40, 40, 40, 40, 40]
            task_deriv_gains: [8, 8, 8, 8, 8, 8]
        operational_space_motion: 
            motion_ctrl_axes: [1, 1, 1, 1, 1, 1]
            task_prop_gains: [1, 1, 1, 1, 1, 1]
            task_deriv_gains: [1, 1, 1, 1, 1, 1]
        open_loop_force: 
            force_ctrl_axes: [0, 0, 1, 0, 0, 0]
        closed_loop_force: 
            force_ctrl_axes: [0, 0, 1, 0, 0, 0]
            wrench_prop_gains: [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
        hybrid_force_motion: 
            motion_ctrl_axes: [1, 1, 0, 1, 1, 1]
            task_prop_gains: [40, 40, 40, 40, 40, 40]
            task_deriv_gains: [8, 8, 8, 8, 8, 8]
            force_ctrl_axes: [0, 0, 1, 0, 0, 0]
            wrench_prop_gains: [0.1, 0.1, 0.1, 0.1, 0.1, 0.1]
train: 
    params: 
        seed: 42
        algo: 
            name: a2c_continuous
        model: 
            name: continuous_a2c_logstd
        network: 
            name: actor_critic
            separate: False
            space: 
                continuous: 
                    mu_activation: None
                    sigma_activation: None
                    mu_init: 
                        name: default
                    sigma_init: 
                        name: const_initializer
                        val: 0
                    fixed_sigma: True
            mlp: 
                units: [256, 128, 64]
                activation: elu
                d2rl: False
                initializer: 
                    name: default
                regularizer: 
                    name: None
        load_checkpoint: False
        load_path: 
        config: 
            name: FactoryTaskNutBoltPick
            full_experiment_name: FactoryTaskNutBoltPick
            env_name: rlgpu
            multi_gpu: False
            ppo: True
            mixed_precision: True
            normalize_input: True
            normalize_value: True
            value_bootstrap: True
            num_actors: 128
            reward_shaper: 
                scale_value: 1.0
            normalize_advantage: True
            gamma: 0.99
            tau: 0.95
            learning_rate: 0.0001
            lr_schedule: fixed
            schedule_type: standard
            kl_threshold: 0.016
            score_to_win: 20000
            max_epochs: 8192
            save_best_after: 50
            save_frequency: 100
            print_stats: True
            grad_norm: 1.0
            entropy_coef: 0.0
            truncate_grads: False
            e_clip: 0.2
            horizon_length: 120
            minibatch_size: 512
            mini_epochs: 8
            critic_coef: 2
            clip_value: True
            seq_len: 4
            bounds_loss_coef: 0.0001
task_name: FactoryTaskNutBoltPick
experiment: 
num_envs: 
seed: 42
torch_deterministic: False
max_iterations: 
physics_engine: physx
pipeline: gpu
sim_device: cuda:0
rl_device: cuda:0
graphics_device_id: 0
num_threads: 4
solver_type: 1
num_subscenes: 4
test: False
checkpoint: 
multi_gpu: False
wandb_activate: False
wandb_group: 
wandb_name: FactoryTaskNutBoltPick
wandb_entity: 
wandb_project: isaacgymenvs
capture_video: False
capture_video_freq: 1464
capture_video_len: 100
force_render: True
headless: True
Setting seed: 42
self.seed = 42
Started to train
Exact experiment name requested from command line: FactoryTaskNutBoltPick
/home/zihan/anaconda3/envs/x/lib/python3.7/site-packages/gym/spaces/box.py:112: UserWarning: WARN: Box bound precision lowered by casting to float32
  logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
Not connected to PVD
+++ Using GPU PhysX
Physics Engine: PhysX
Physics Device: cuda:0
GPU Pipeline: enabled
Using SDF cache directory '/home/zihan/.isaacgym/sdf_V100'
~!~!~! Loaded/Cooked SDF triangle mesh 0 @ 0x55f2e8c6b7a0, resolution=256, spacing=0.000108
  ~!~! Bounds:  (-0.012000, 0.012000) (-0.013856, 0.013856) (0.016000, 0.029000)
  ~!~! Extents: (0.024000, 0.027712, 0.013000)
  ~!~! Resolution: (222, 256, 121)
[Error] [carb.gym.plugin] Failed to create SDF triangle mesh
Segmentation fault (core dumped)

Any idea of why this is happening?

Thanks!

Number of environments changes during training for AMP example

I think I found an issue in your AMP implementation in Isaac, I am not entirely sure though. For some reason the number of environments changes every second or third step randomly during training and is not the set 4096 anymore. I checked that using the size of the observation vector which seems to change:

Inside humanoid_amp_base.py:
[4096, 105]
[440, 105]
[4096, 105]
[487, 105]
[4096, 105]
[457, 105]
[4096, 105]

Inside humanoid_amp.py:
[4096, 105]
[440, 105]
[440, 105]
[4096, 105]
[487, 105]
[487, 105]
[4096, 105]
[457, 105]
[457, 105]
[4096, 105]

This makes randomization of variables that are supposed to be the same over the episode but different for every environment difficult. At least I am not exactly sure what is happening... Is there an explanation for this? This only happens during training, and only for the AMP example. It does not happen during inference or for other examples.

Do we able to use image as input for IsaacGym ?

Dear, author:
we have seen resnet_actor_critic setting for training the RL algorithm. Does IsaacGym support image type input ? Thanks advance. looking forward your reply !

Adding custom tabs and properties to the Viewer GUI

Is there a way to add custom tabs and properties to the Viewer GUI? Also how can I change the background color from black to another one? Can we add a texture to it?

Multi-agent reinforcement learning algorithms

I'm trying to use Isaac Gym for multi-agent reinforcement learning (MARL), is there any future plan on this?

If not, can you give any suggestions on how should I integrate MARL algorithms into Issac Gym? Should I start from rl_games repo or maybe integrate other MARL-supported repos?

Thanks for your help!

pip install -e . doesn't install hydra

ModuleNotFoundError: No module named 'hydra'

Found issue on my side

Observation Space for Ant

Hi,

I'm trying to understand which observations correspond to the foot position and orientation for each of the four feet for the ant environment. Specifically, I want to know if, for each timestep, which foot (feet) are in contact with the ground (I don't think this is being calculated anywhere, but could be wrong?). If it's not being calculated, I would also need to know the length of the limb corresponding to "foot" so I can calculate if the end of this limb intersects with the ground plane or not.

More generally, it would be useful if there was documentation on the components of these observation spaces (i.e. torso_pos, torso_vel, leg_pos, leg_rot, etc) and their dimensionality for each of the environments.

Thanks!

Documentation should be updated

Hi,

The documentation, e.g. in https://github.com/NVIDIA-Omniverse/IsaacGymEnvs/blob/main/docs/framework.md should probably be updated.

How is quaternion that returned by gym.acquire_rigid_body_state_tensor defined?

I need positions and rotations of rigid body states of Humanoid. But I wonder if positions and rotations are defined relative to the root of humanoid(like Pelvis?) or the world?
It puzzled me a lot when trying to convert the behavior of humanoid to a <.bvh> file. Any suggestions for converting into .bvh?

Readiness and hard constraints

Hi there,

when will gym be ready for primetime?
I am still with Isaac Gym on Sim and wondering when I should switch to Omniverse?

Second question:
How hard can constraints be made?
I don't know if I am doing something wrong here but no matter whether I am using Attractors or concerning the maximum force (effort) of a joint, there seem to be a reaction time until the constraint solving force is applied, meaning the constraints seem too "soft". What am I doing wrong :)

E.g. I want to have 2 actors be attached to each other but the spring (stiffness) between them seems too soft even if I set the attractor's stiffness up to 1e10 or more, same goes for the servos DOF stiffness and effort, please see the video.
On the video you can see the problem I am facing: Since I cannot fixate the different parts of the finger, I try to make the hand more stiff but without success so far.

Video

Kind regards

KeyError when loading usd in franka_cube_ik_osc.py example

The documentation says the following:

The load_asset method uses the file name extension to determine the asset file format. Supported extensions include .urdf for URDF files, .xml for MJCF files, and .usd/.usda for USD files.

However when I change the asset in franka_cube_ik_osc.py example to "urdf/franka_description/robots/franka_panda/franka_panda.usd" I get the following error:

Traceback (most recent call last): File "franka_cube_ik_osc.py", line 208, in <module> franka_hand_index = franka_link_dict["panda_hand"] KeyError: 'panda_hand'

Is there something in the example code that is not compatible with usd format?

PyPI Release

Hello, this is a really cool project! Would you be up to making a PyPI release so that people can install via pip install issacgym? It pins the version which will also help people reproduce results in the future :D

RuntimeError: Failed to acquire interface: carb::gym::Gym (pluginName: nullptr)

Hi, isaacgym can not run in A100-SXM-80GB ?
When I run the test code in this type of GPUs, I get the following note:

Exact experiment name requested from command line: Ant
/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/gym/spaces/box.py:84: UserWarning: WARN: Box bound precision lowered by casting to float32
logger.warn(f"Box bound precision lowered by casting to {self.dtype}")
[Error] [carb] Failed to acquire interface: [carb::gym::Gym v0.1], by client: carb.gym.python.gym_37 (plugin name: (null))
Error executing job with overrides: []
Traceback (most recent call last):
File "train.py", line 127, in launch_rlg_hydra
'play': cfg.test,
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/torch_runner.py", line 139, in run
self.run_train()
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/torch_runner.py", line 122, in run_train
agent = self.algo_factory.create(self.algo_name, base_name='run', config=self.config)
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/common/object_factory.py", line 15, in create
return builder(**kwargs)
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/torch_runner.py", line 23, in
self.algo_factory.register_builder('a2c_continuous', lambda **kwargs : a2c_continuous.A2CAgent(**kwargs))
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/algos_torch/a2c_continuous.py", line 18, in init
a2c_common.ContinuousA2CBase.init(self, base_name, config)
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/common/a2c_common.py", line 980, in init
A2CBase.init(self, base_name, config)
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/common/a2c_common.py", line 85, in init
self.vec_env = vecenv.create_vec_env(self.env_name, self.num_actors, **self.env_config)
File "/home/yangchao/anaconda3/envs/rl/lib/python3.7/site-packages/rl_games/common/vecenv.py", line 282, in create_vec_env
return vecenv_config[vec_env_name](config_name, num_actors, **kwargs)
File "train.py", line 95, in
lambda config_name, num_actors, **kwargs: RLGPUEnv(config_name, num_actors, **kwargs))
File "/home/yangchao/Workspace/1-isaacgym/IsaacGymEnvs/isaacgymenvs/utils/rlgames_utils.py", line 159, in init
self.env = env_configurations.configurations[config_name]'env_creator'
File "train.py", line 98, in
'env_creator': lambda **kwargs: create_rlgpu_env(**kwargs),
File "/home/yangchao/Workspace/1-isaacgym/IsaacGymEnvs/isaacgymenvs/utils/rlgames_utils.py", line 91, in create_rlgpu_env
headless=headless
File "/home/yangchao/Workspace/1-isaacgym/IsaacGymEnvs/isaacgymenvs/tasks/ant.py", line 70, in init
super().init(config=self.cfg, sim_device=sim_device, graphics_device_id=graphics_device_id, headless=headless)
File "/home/yangchao/Workspace/1-isaacgym/IsaacGymEnvs/isaacgymenvs/tasks/base/vec_task.py", line 181, in init
self.gym = gymapi.acquire_gym()
RuntimeError: Failed to acquire interface: carb::gym::Gym (pluginName: nullptr)
Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

nvidia-omniverse / isaacgymenvs Goto Github PK

isaacgymenvs's People

Stargazers

Watchers

Forkers

isaacgymenvs's Issues

Recommend Projects

Recommend Topics

Recommend Org