pantor / learning-pick-and-place Goto Github PK

Hi,
Thank you for sharing the great work. I have an idea of training this in a simulation environment (Isaac Sim) and then transferring it to the real robot. Do you think it is worth a try or it is better to train it directly on the real robot?

Unable to locate terminal-setup.sh. Where is it supposed to be located?

moveit.launch does not appear to be part of /launch.

We have been able to get a bit further but keep running in to different errors with python modules.
The first ModuleNotFoundError was:

root@4a18271afcb4:/home/Workspace/catkin_ws# rosrun bin_picking grasping.py
Traceback (most recent call last):
  File "/home/Workspace/catkin_ws/src/learning-pick-and-place/scripts/grasping.py", line 9, in <module>
    from utils.camera import Camera
  File "/home/Workspace/catkin_ws/src/learning-pick-and-place/scripts/utils/camera.py", line 12, in <module>
    from orthographical import OrthographicImage
ModuleNotFoundError: No module named 'orthographical'

This was solved by exporting the path to the orthographical.cpython-36m-x86_64-linux-gnu.so by export PYTHONPATH=$PYTHONPATH:/home/Workspace/catkin_ws/devel/lib

Next ModuleNotFoundError is where we are stuck now:

root@4a18271afcb4:/home/Workspace/catkin_ws# rosrun bin_picking grasping.py
2020-11-03 11:44:52.218547: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /home/Workspace/catkin_ws/devel/lib:/opt/ros/kinetic/lib:/opt/ros/kinetic/lib/x86_64-linux-gnu
2020-11-03 11:44:52.218649: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
  File "/home/Workspace/catkin_ws/src/learning-pick-and-place/scripts/grasping.py", line 12, in <module>
    from agents.agent_place import Agent as AgentPlace
  File "/home/Workspace/catkin_ws/src/learning-pick-and-place/scripts/agents/agent_place.py", line 12, in <module>
    from inference.planar import InferencePlanarPose
ModuleNotFoundError: No module named 'inference.planar'

The planar.py seems to be missing from scripts/inference. Would you be able to push the missing files to your repo?

Hello Pantor,

thanks for sharing your very exciting work and code! Is it possible to adapt your code to a KUKA iiwa? What steps would be necessary here? Replacing the launch and include files would be my guess. Are there also other interfaces that I need to adapt for this. I would be glad about the help and answer. I would like to test and develop your code for other applications.

Best regards,
Marcel

In your README you write:

"Patching CvBridge for Python3 and CMake >= 3.12 is given by a snippet in GitLab."

Could you provide a link to the snippet you used? I assume that it's located on your local git server? Currently we're simply cloning vision_opencv (v. 1.12.8) into our repo and building that from source.