prophesee-ai / prophesee_ros_wrapper Goto Github PK

ROS driver for Prophesee event-based sensors

License: Apache License 2.0

CMake 7.58% C++ 92.42%

event-based machine-vision high-speed-data-acquisition low-power bio-inspired ros-driver ros atis change-detection prophesee

prophesee_ros_wrapper's Introduction

Prophesee ROS Wrapper

The aim of this metapackage is wrapping event-based data from Prophesee sensors using OpenEB software and publishing the event-based data to ROS.

This metapackage contains ROS wrapper (prophesee_ros_driver) and messages (prophesee_event_msgs) for Prophesee sensors.

prophesee_ros_driver is a ROS wrapper including the following nodes:

prophesee_ros_publisher - publishing data from Prophesee sensor to ROS topics
prophesee_ros_viewer - listening data from ROS topics and visualizing them on a screen

prophesee_event_msgs package contains ROS message types for Prophesee event-based data, including:

Event - an event from a Prophesee camera (uint16 x, uint16 y, ros::Time ts, bool polarity)
EventArray - a buffer of events (Event[] events)

Supported Prophesee Evaluation Kit Cameras :

EVKV2 Gen4.1
EVKV3 Gen3.1, Gen4.1
EVKV4 IMX636

Requirements

Ubuntu 20.04 or 22.04
ROS Noetic or ROS Melodic
OpenEB - starting from v2.2.0

Installation

First, retrieve and compile OpenEB.

Then, compile the packages:

Clone the source code to your catkin workspace (create a workspace, if needed)

    cd catkin_ws/src
    git clone https://github.com/prophesee-ai/prophesee_ros_wrapper.git
    cd ..

Compile
```
    catkin_make
```

Source the workspace

    source ~/catkin_ws/devel/setup.bash

Getting Started

Publishing data from a camera and listening to them

To publish data from Prophesee camera to ROS topics, run:

      roslaunch prophesee_ros_driver prophesee_publisher.launch

The following topics will be published:

/prophesee/camera/cd_events_buffer - buffer of CD (Change Detection) events
/prophesee/camera/camera_info - information about the camera

To listen data from ROS topics and visualize them:

      roslaunch prophesee_ros_driver prophesee_viewer.launch

Recording data from a live camera to rosbag

To record data from live camera to rosbag:

Start the publisher:

      roslaunch prophesee_ros_driver prophesee_publisher.launch

Start rosbag recording (choose the topics to record or record all available topics):

      rosbag record -a

Publishing data from a RAW file

To publish data from RAW file to ROS topics and view the data:

Update the prophesee_publisher.launch file to set the path to your RAW file (in raw_file_to_read parameter)

      rosed prophesee_ros_driver prophesee_publisher.launch

Start the ROS core

      roscore

Start the viewer, at first to be sure to not miss any data:

      roslaunch prophesee_ros_driver prophesee_viewer.launch

Start the publisher:

      roslaunch prophesee_ros_driver prophesee_publisher.launch

At the end of the RAW file, the publisher will stop on its own, but the viewer won't stop, so it's up to you to quit the viewer.

Recording data from a RAW file to rosbag

To record data from RAW file to rosbag:

Update the prophesee_publisher.launch file to set the path to your RAW file (in raw_file_to_read parameter)

      rosed prophesee_ros_driver prophesee_publisher.launch

Start the ROS core

      roscore

Start rosbag recording (choose the topics to record or record all available topics):

      rosbag record -a

Start the publisher:

      roslaunch prophesee_ros_driver prophesee_publisher.launch

Going Further

The ROS wrapper we propose in this repo is a minimal example to get you started. There are multiple ways to enhance it in terms of features and performance. The code is open to contributions, thus do not hesitate to propose pull requests.

We also recommend you to check the metavision_ros_driver repo from berndpfrommer that offers a ROS driver for event-based cameras with some speed and features enhancements.

Contact

In case of any issue, please raise an issue here on GitHub. For any other information contact us.

prophesee_ros_wrapper's People

Contributors

Stargazers

Watchers

prophesee_ros_wrapper's Issues

How to change biases

Is it possible to change camera biases for a Prophesee camera using the ROS driver? Currently I need to terminate my ROS program, launch the metavision player then restart ROS again to see the effects of the new biases.

Unable to locate package prophesee-*

Following the instructions in the prophesee knowledge center, I tried to install the prophesee driver sdk for Linux. My system is a Jetson Nano running Ubuntu 18.04 64 bit.
When I reached the step:
$ sudo apt install prophesee-* metavision-*

I get errors:
E: Unable to locate package prophesee-*
E: Couldn't find any package by glob 'prophesee-'
E: Couldn't find any package by regex 'prophesee-'

Any advice?

Question about the Evaluation Kit

Hi,

we plan to buy a Metavision Evaluation Kit 3 - Gen4H - HD for research. But before that, we want to know that when it provides the event data, does it comes with the relative frame image?

Thank you very much !

Frame rate gets slower and slower when run prophesee_viewer.launch

My Prophesee Evaluation Kit Cameras is EVKV3 Gen4.1，when I run:
roslaunch prophesee_ros_driver prophesee_viewer.launch
the frame rate in the CD Event Window will get slower and slower,why?

IMU readings

Hiya,
I can see that IMU readings used to be part of the code, but then were removed. We'd like to use IMU, could it please be added back?

Many thanks,
Tobias
/cc @fervanegas

ROS2 build script

Is there a build script or process for using this wrapper with ROS2? Or is there another ROS2 version? Thanks very much.

Plan to port nodes to nodelets

Hi,

Is there any plan to port nodes to nodelets to improve the performance?

Best,
Jiawei

Couldn't able to find the Prophesee SDK

For some reason, this command is not working?

sudo apt install prophesee-*

Could you please help me with this?

libmetavision_hal_discovery.so.2: cannot open shared object file:

Hi,

I have a CenturyArk SilkyEvCam v02.01.04 and installed the metavision essentials and the metavision HAL plugin for this camera. When I run the command metavision_viewer the camera is correctly loaded and a window showing the incoming events is shown. Then I compiled and installed openEB following the instructions in github. Following I cloned and compiled this ros wrapper and the process went without errors. However, when I try to:

roslaunch prophesee_ros_driver prophesee_publisher.launch

I get the following error:

unloading/devel/lib/prophesee_ros_driver/prophesee_ros_publisher: error while loading shared libraries: libmetavision_hal_discovery.so.2: cannot open shared object file: No such file or directory

Could you please guide me on how to get the ros driver to work with the SilkyEvCam? Thanks.

Some more info:

$dmesg
[ 3298.100458] usb 2-3: reset SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[ 3298.120768] usb 2-3: LPM exit latency is zeroed, disabling LPM.
[ 5345.469989] usb 2-3: USB disconnect, device number 2
[ 5349.337972] usb 1-4: new high-speed USB device number 12 using xhci_hcd
[ 5349.581049] usb 1-4: New USB device found, idVendor=31f7, idProduct=0002, bcdDevice= 0.00
[ 5349.581054] usb 1-4: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 5349.581057] usb 1-4: Product: SilkyEvCam v02.01.04
[ 5349.581060] usb 1-4: Manufacturer: CenturyArks

$uname -a
Linux lab 5.4.0-80-generic #90~18.04.1-Ubuntu SMP Tue Jul 13 19:40:02 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

$metavision_hal_ls
Device detected: CenturyArks:evc3a_plugin_gen31:0000000000000034

$ metavision_platform_info 

------------------------------------------ 
PLATFORM INFORMATION 
------------------------------------------ 


####  DISTRIBUTION  #### 

Linux labu 5.4.0-80-generic #90~18.04.1-Ubuntu SMP Tue Jul 13 19:40:02 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux 

####  OPENGL  #### 

OpenGL vendor string: NVIDIA Corporation 
OpenGL renderer string: GeForce RTX 2060 SUPER/PCIe/SSE2 
OpenGL core profile version string: 4.6.0 NVIDIA 440.33.01 
OpenGL core profile shading language version string: 4.60 NVIDIA 
OpenGL core profile context flags: (none) 
OpenGL core profile profile mask: core profile 
OpenGL core profile extensions: 
OpenGL version string: 4.6.0 NVIDIA 440.33.01 
OpenGL shading language version string: 4.60 NVIDIA 
OpenGL context flags: (none) 
OpenGL profile mask: (none) 
OpenGL extensions: 
OpenGL ES profile version string: OpenGL ES 3.2 NVIDIA 440.33.01 
OpenGL ES profile shading language version string: OpenGL ES GLSL ES 3.20 
OpenGL ES profile extensions: 

####  VIRTUAL MACHINE  #### 

Virtual Machine:              NO
Docker:                       NO

------------------------------------------ 
METAVISION SOFTWARE INFORMATION 
------------------------------------------ 


####  INSTALLED SOFTWARE  #### 

Version:                      2.2.2.2120809
Id:                           11a19a7899e5fc8dcc27add6e273a66e513afa35

####  INSTALLED PACKAGES - METAVISION  #### 

Name                               Version   Architecture   Description         
metavision-hal-prophesee-plugins   2.2.2     amd64          Prophesee Plugins for Metavision HAL.

------------------------------------------ 
METAVISION SYSTEMS INFORMATION 
------------------------------------------ 


####  SYSTEMS AVAILABLE  #### 

# FOUND CenturyArks GEN 3.1 VGA # 

Connection                    USB
FW Build Date                 Fri Jan 15 17:26:36 2021
FW Release Version            2.1.4
Integrator                    CenturyArks
Raw Formats                   EVT3
Sensor Info                   3.1
Serial                        0000000000000034
System Build Date             Sat Dec 19 02:49:38 2020
System Version                4.1.1
System Version Control ID     0x0
SystemID                      40

DEFAULT BIASES 
bias_diff                     299
bias_diff_off                 221
bias_diff_on                  384
bias_fo                       1477
bias_hpf                      1499
bias_pr                       1250
bias_refr                     1500


####  SYSTEM'S USB PORTS  #### 

No systems USB connected have been found on your platform.

$ metavision_software_info -v -c
11a19a7899e5fc8dcc27add6e273a66e513afa35 
2.2.2.2120809

Camera Biases

Hi
Is it possible to change the biases of a specific model of prophesee cameras. I know the bias values that I use in the metavision software but how do I use the same ones in ROS.

Converting rosbag to .raw format that is compatible with metavision

Hello,
I am working with a GEN3 event camera for recording data from a traffic scenario and multiple sensors to rosbag. I would like to convert the event-based camera information from the rostopic to .raw format which is compatible with metavision. Is there a way to do that?

Publish greyscale image error

When I launch the prophesee_publisher file, the greyscale image is not published even if I changed to like this ()

I checked the publishGrayLevels function in source code and raise the try catch error.

Please let me know if there are my mistakes.