Generating segmentation masks and bounding box from replica about blenderproc HOT 4 CLOSED

Hey,

thanks for your interest in BlenderProc!

For the Replica Dataset do we currently not support semantic segmentation and the color is only vertex color. The reason for that is that the semantic segmentation info and the correct color info are not easily extractable out of the provided files in the Replica dataset.

But I would suggest you might use 3D-Front or SUNCG, there we already support semantic segmentation.

All objects in a scene need to have the custom property cp_category_id to work correctly in such a setting.

Does this help you?

PS: The objects in the replica dataset are also not split up. To get better support their we would need a separated mesh from the makers of that dataset, which they currently do not provide.

Best,
Max

from blenderproc.

Hey Max,

Thank you so much for letting me know this.

Unfortunately, I need to generate the data specifically for Replica only. Do you think it's possible to generate all the fields that I want from replica (maybe using other things like habitat-API) or is there a fundamental issue in replica which makes it impossible to generate the requisite data?

Best,
Ayush

from blenderproc.

Hey Ayush,

as far as I know semantic segmentation and color are supported by the replica software tool box. But I don’t know if they support depth, bounding boxes or setting specific internal camera intrinsic.

Of course, if it would be possible to separate the objects in the replica dataset and assign them a correct custom property than our code would work. But we couldn’t make it work, maybe you have more luck.

Else I would try to look at the habitat API, it is also from Facebook so they have it easier to integrate replica, which is also from Facebook.

Best regards,
Max

from blenderproc.

Thank you Max for all your help! Closing this issue for now.

from blenderproc.