Code Monkey home page Code Monkey logo

bpycv's Introduction

bpycv: computer vision and deep learning utils for Blender

Contents: Features | Install | Fast Demo | Tips

0
render instance annoatation, RGB image and depth in one line code

▮ Features

  • Render depth
  • Render annotations for semantic segmentation, instance segmentation, and panoptic segmentation
  • Generate 6DoF pose GT
  • Pre-define domain randomization: light
  • Pre-define domain randomization: background
  • Pre-define domain randomization: distractor
  • Pre-define domain randomization: textures
  • To Cityscapes format
  • To COCO format

News: We win 🥈2nd place in IROS 2020 Open Cloud Robot Table Organization Challenge (OCRTOC)

▮ Install

bpycv support Blender 2.8, 2.9

Troubleshooting tips: doc/troubleshooting.md

1. Install OpenExr

bpycv use OpenExr to extract depth map from Blender

For a Debian-based Linux(Ubuntu):

sudo apt-get install libopenexr-dev

For other OS, please follow OpenExr's instruction.

2. Install python package

Example for Blender 2.90:

cd <path to blender>/2.90/python/bin
./python3.7m -m ensurepip  # get pip
./python3.7m -m pip install -U pip setuptools wheel 
./python3.7m -m pip install -U opencv-python openexr bpycv

▮ Fast Demo

1. Instance Segmentation and Depth Demo

Copy-paste this code to Scripting/Text Editor and click Run Script button(or Alt+P)

import cv2
import bpy
import bpycv
import random
import numpy as np

# remove all MESH objects
[bpy.data.objects.remove(obj) for obj in bpy.data.objects if obj.type == "MESH"]

for index in range(1, 20):
    # create cube and sphere as instance at random location
    location = [random.uniform(-2, 2) for _ in range(3)]
    if index % 2:
        bpy.ops.mesh.primitive_cube_add(size=0.5, location=location)
        categories_id = 1
    else:
        bpy.ops.mesh.primitive_uv_sphere_add(radius=0.5, location=location)
        categories_id = 2
    obj = bpy.context.active_object
    # set each instance a unique inst_id, which is used to generate instance annotation.
    obj["inst_id"] = categories_id * 1000 + index

# render image, instance annoatation and depth in one line code
# result["ycb_meta"] is 6d pose GT
result = bpycv.render_data()

# save result
cv2.imwrite(
    "demo-rgb.jpg", result["image"][..., ::-1]
)  # transfer RGB image to opencv's BGR

# save instance map as 16 bit png
# the value of each pixel represents the inst_id of the object to which the pixel belongs
cv2.imwrite("demo-inst.png", np.uint16(result["inst"]))

# convert depth units from meters to millimeters
depth_in_mm = result["depth"] * 1000
cv2.imwrite("demo-depth.png", np.uint16(depth_in_mm))  # save as 16bit png

# visualization inst_rgb_depth for human
cv2.imwrite("demo-vis(inst_rgb_depth).jpg", result.vis()[..., ::-1])

Open ./demo-vis(inst_rgb_depth).jpg:

demo-vis(inst_rgb_depth)

2. YCB Demo

Inculding domain randomization for background, light, and distractor.

mkdir ycb_demo
cd ycb_demo/

# prepare demo code and data
git clone https://github.com/DIYer22/bpycv
git clone https://github.com/DIYer22/bpycv_example_data

cd bpycv/example/

blender -b -P ycb_demo.py

cd dataset/vis/
ls .  # visualize result here

Open 0.jpg:
0
(instance_map | RGB | depth)

YCB demo code: example/ycb_demo.py

3. 6DoF Pose Demo

Generate and visualize 6DoF pose GT: example/6d_pose_demo.py

▮ Tips

Blender may can't direct load .obj and .dea file from YCB and ShapeNet dataset.
It's better to transefer and format using meshlabserver by run meshlabserver -i raw.obj -o for_blender.obj -m wt



suggestion and pull request are welcome 😊

bpycv's People

Contributors

diyer22 avatar avinash2468 avatar salingo avatar shashwatnigam99 avatar ethnhe avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.