isrkhou / physim-dataset-generator Goto Github PK

This repository implements a software tool for synthesizing images of physically realistic cluttered scenes using 3D CAD models as described in our paper:

By Chaitanya Mitash, Kostas Bekris, Abdeslam Boularias (Rutgers University).

To appear at the IEEE International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, 2017.

To cite the work:

@article{physim,
  title={A Self-supervised Learning System for Object Detection using Physics Simulation and Multi-view Pose Estimation},
  author={Mitash, Chaitanya and Bekris, Kostas and Boularias, Abdeslam},
  journal={arXiv:1703.03347},
  year={2017}
}

1. Download and extract Blender
2. In ~/.bashrc, add line export BLENDER_PATH=/path/to/blender/blender
3. Get the get-pip.py file from the pip documentation
4. Install the yaml package in the python packaged with blender using the commands below.

$ /path/to/blender/2.xx/python/bin/python3.5m /path/to/get-pip/get-pip.py
$ /path/to/blender/2.xx/python/bin/python3.5m /blender-version/2.xx/python/bin/pip install pyyaml

1. In ~/.bashrc, add line export PHYSIM_GENDATA=/path/to/repo.
2. Rename config.yml.shelf or config.yml.shelf to config.yml and modify simulation parameters if required.
3. Run python generate_pictures.py
4. The generated data can be found in the folder rendered_images. Available environments are table and shelf.

1. Images of scenes.
2. Labeled bounding box files for each scene <label, tl_x, tl_y, br_x, br_y> or if the pixel label mode is selected, a pixel-wise labeled image is generated for each scene where the pixel value is the ground-truth class value.
3. Debug images indicating the bounding-boxes over the objects.
4. .blend files to debug the simulation parameters.

the example cfg files contain the parameters of simulation.

camera:
  num_poses: <number of views to render from>
  camera_poses: [[pos_x, pos_y, pos_z, quat_w, quat_x, quat_y, quat_z], ...]
  camera_intrinsics: [[f_x, 0.0, c_x],[0.0, f_y, c_y],[0.0, 0.0, 1.0]]

rest_surface:
  type: shelf
  surface_pose: [pos_x, pos_y, pos_z, quat_w, quat_x, quat_y, quat_z]

Models: [model_1, model_2, ...]

params:
  num_images: <number of training images>
  label_type: box
  minimum_objects_in_scene: <minimum object per scene>
  maximum_objects_in_scene: <minimum object per scene>
  range_x: [<min_x>, <max_x>]
  range_y: [<min_y>, <max_y>]
  range_z: [<min_z>, <max_z>]
  num_simulation_steps: <number os simulation steps to run>
  light_position_range_x: [<min_x>, <max_x>]
  light_position_range_y: [<min_y>, <max_y>]
  light_position_range_z: [<min_z>, <max_z>]