Hardware-accelerated 3D scene reconstruction and Nav2 local costmap provider using nvblox
Home Page: https://developer.nvidia.com/isaac-ros-gems
License: Apache License 2.0
Hi, I was trying to use nvblox ros with ubuntu 18.04 and realsene d455 camera. I installed ros-foxy from source, installed isaac_nvblox_ros and its dependencies along with realsense2_ros. I think I supplied correct camera topics/transform and can run both but have two problems.
As is, the nvblox_ros stops with SIGSEGV. I noticed that thrust::transform function in conversions.cu that divides the depth values by 1000 in RosConverter::depthImageFromImageMessage() is the cause (because I can use print commands up to that point) . The full error I get is: ***aborted at 1649440186 (unix time) try date -d 1649440186 if you are using GNU date.. 0x0 unkonwn, SIGSEGV received by PID 11109(TID 0x7fb5aba010) from PID 33530880; stack trace ***
I suppressed that line (thrust::transform()) just to see what happens. Now it runs and If I do "ros2 topic hz /nvblox_node/mesh" it seems to publish on to that topic. Same is with the pointcloud topic. However, there is no data in the fields when I echo the topics. I cant see it in rviz because I get the error: " [rviz2]: could not load pixmap package://nvblox_ros_plugin/icons/classes/NvbloxMesh.png". I do have sourced the ros and nvblox bash files before running rviz.
Can anyone please help with this or let me know if you need more information. Note that I am not running nav2_bringup and have suppressed the corresponding node in the carter_sim launch file. have also changed global frame in the nvblox-yaml file as per what I have from realsense2_ros. Just one thing, the ubuntu 18.04 that I have has jetpack 4.6 in case that is the root cause.
What should I do when I get the following error while running./scripts/run_dev.sh?
failed to copy: httpReadSeeker: failed open: failed to authorize: rpc error: code = Unknown desc = failed to fetch anonymous token: unexpected status: 401 Unauthorized
The pipeline seems to support RGB-D cameras quite well. That being said, how difficult would it be to adapt it to ingest point clouds (which may or may not contain RGB information) instead? I'm thinking something similar to what is possible with voxblox.
I'm working with a native installation in x64 20.04, d435i. As AI modules, nvblox and nav2 won't work, as it sends errors similar to this one, doesn't matter what topics a put, if use / symbol or not as /camera/...
[nvblox_node-2] @ 0x55d6f42827be _start
[ERROR] [nvblox_node-2]: process has died [pid 43249, exit code -11, cmd '/home/igcs/isaac_ws/install/nvblox_ros/lib/nvblox_ros/nvblox_node --ros-args --params-file /home/igcs/isaac_ws/install/nvblox_nav2/share/nvblox_nav2/params/igcs_nvblox.yaml --params-file /tmp/launch_params_s3ofgvz8 -r depth/image:=camera/depth/image_rect_raw -r depth/camera_info:=camera/depth/camera_info -r color/image:=camera/color/image_raw -r color/camera_info:=camera/color/camera_info'].
[rviz2-1] [INFO] [1649357703.312523677] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1649357703.312730329] [rviz2]: OpenGl version: 4.6 (GLSL 4.6)
[rviz2-1] [INFO] [1649357703.350449625] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1649357703.518772294] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1649357703.533156402] [rviz2]: Stereo is NOT SUPPORTED
[rviz2-1] [INFO] [1649357703.551877001] [rviz2]: Stereo is NOT SUPPORTED
it is like hardcoded to it only works only in Isaac Sim. I guess in the nvblox_node file. Little to little I was removing error after changing things, but this ends here. It looks Nav2 and vBlox only works in Sim I guess.
VIO is the only that works. I can access to landmarks point, observations points with IMU activated.
Recently I made own environment in isaac sim for nvblox, it working so fine.
Now I want to using multi robots with nvblox in Isaac Sim.
Like multi-robot navigation in Isaac Sim doc, I'm just adding namespace(carter0) and remapping topics from carter_sim.launch.py.
# SPDX-FileCopyrightText: NVIDIA CORPORATION & AFFILIATES
# Copyright (c) 2021-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# SPDX-License-Identifier: Apache-2.0
from email.headerregistry import Group
import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument, IncludeLaunchDescription
from launch.conditions import IfCondition
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch.substitutions import LaunchConfiguration
from launch.actions import GroupAction
from launch_ros.actions import PushRosNamespace
from launch_ros.actions import Node
namespace = LaunchConfiguration("namespace")
use_namespace = LaunchConfiguration("use_namespace")
use_lidar = LaunchConfiguration("use_lidar")
use_depth = LaunchConfiguration("use_depth")
def generate_launch_description():
###
declare_namespace_cmd = DeclareLaunchArgument("namespace", default_value="carter0", description="Top-level namespace")
declare_use_namespace_cmd = DeclareLaunchArgument(
"use_namespace", default_value="true", description="Whether to apply a namespace to the navigation stack"
)
###
params_file_arg = DeclareLaunchArgument(
'params_file', default_value=os.path.join(
get_package_share_directory(
'nvblox_nav2'), 'params/warehouse', 'multi_robot_carter_navigation_params_0.yaml'),
description='Full path to param file to load')
use_sim_time_arg = DeclareLaunchArgument(
'use_sim_time', default_value='True',
description='Use simulation (Omniverse Isaac Sim) clock if true')
nvblox_param_arg = DeclareLaunchArgument(
'nvblox_params_file',
default_value=os.path.join(
get_package_share_directory('nvblox_nav2'), 'params/warehouse', 'nvblox_0.yaml'
)
)
use_depth_arg = DeclareLaunchArgument(
'use_depth', default_value='True',
description='Use depth as an input for nvblox reconstruction'
)
use_lidar_arg = DeclareLaunchArgument(
'use_lidar', default_value='False',
description='Use lidar as an input for nvblox reconstruction'
)
run_rviz_arg = DeclareLaunchArgument(
'run_rviz', default_value='True',
description='Whether to start RVIZ')
run_nav2_arg = DeclareLaunchArgument(
'run_nav2', default_value='True',
description='Whether to run nav2')
nav2_bringup_launch_dir = os.path.join(
get_package_share_directory('nav2_bringup'), 'launch')
rviz_config_dir = os.path.join(get_package_share_directory(
'nvblox_nav2'), 'config', 'carter_navigation_namespaced.rviz')
rviz_launch = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(nav2_bringup_launch_dir, 'rviz_launch.py')),
launch_arguments={'using_namespace': 'True',
'autostart': 'True',
'rviz_config': rviz_config_dir}.items(),
condition=IfCondition(LaunchConfiguration('run_rviz')))
nav2_launch = IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(nav2_bringup_launch_dir, 'navigation_launch.py')),
launch_arguments={'using_namespace': 'True',
'use_sim_time': LaunchConfiguration('use_sim_time'),
'params_file': LaunchConfiguration('params_file'),
'autostart': 'True'}.items(),
condition=IfCondition(LaunchConfiguration('run_nav2')))
nvblox_node = Node(
package='nvblox_ros', executable='nvblox_node',
parameters=[LaunchConfiguration('nvblox_params_file'),{
'use_sim_time': LaunchConfiguration('use_sim_time'),
'use_depth': LaunchConfiguration('use_depth'),
'use_lidar': LaunchConfiguration('use_lidar')}],
output='screen',
remappings=[('depth/image', 'left/depth'),
('depth/camera_info', 'left/camera_info'),
('color/image', 'left/rgb'),
('color/camera_info', 'left/camera_info'),
('pointcloud', 'point_cloud'),
('/tf', 'tf'),
('/tf_static', 'tf_static'),
])
map_to_odom_publisher = Node(
package='tf2_ros', executable='static_transform_publisher',
parameters=[{'use_sim_time': LaunchConfiguration('use_sim_time')}],
output='screen',
arguments=['0.0', '0.0', '-0.3', '0.0', '0.0', '0.0', 'map',
'odom'],
remappings=[('/tf', '/carter0/tf'),
('/tf_static', '/carter0/tf_static'),
('map', '/carter0/map'),
('odom','/carter0/odom')],
condition=IfCondition(LaunchConfiguration('run_nav2')))
carter0_nav2_namespaced = GroupAction(
actions=[
PushRosNamespace('carter0'),
nav2_launch, map_to_odom_publisher,nvblox_node,
]
)
return LaunchDescription([
declare_namespace_cmd,
declare_use_namespace_cmd,
params_file_arg,
use_sim_time_arg,
nvblox_param_arg,
use_depth_arg,
use_lidar_arg,
run_rviz_arg,
run_nav2_arg,
rviz_launch,
#nav2_launch,
carter0_nav2_namespaced,
#nvblox_node,
#map_to_odom_publisher
])now I get list from ros2 topic list:
/carter0/behavior_server/transition_event
/carter0/behavior_tree_log
/carter0/bond
/carter0/bt_navigator/transition_event
/carter0/clicked_point
/carter0/clock
/carter0/cmd_vel
/carter0/cmd_vel_nav
/carter0/controller_server/transition_event
/carter0/cost_cloud
/carter0/evaluation
/carter0/global_costmap/costmap
/carter0/global_costmap/costmap_raw
/carter0/global_costmap/costmap_updates
/carter0/global_costmap/footprint
/carter0/global_costmap/global_costmap/transition_event
/carter0/global_costmap/published_footprint
/carter0/goal_pose
/carter0/initialpose
/carter0/left/camera_info
/carter0/left/depth
/carter0/left/rgb
/carter0/local_costmap/costmap
/carter0/local_costmap/costmap_raw
/carter0/local_costmap/costmap_updates
/carter0/local_costmap/footprint
/carter0/local_costmap/local_costmap/transition_event
/carter0/local_costmap/published_footprint
/carter0/local_plan
/carter0/marker
/carter0/nvblox_node/map_slice
/carter0/nvblox_node/map_slice_bounds
/carter0/nvblox_node/map_slice_bounds_array
/carter0/nvblox_node/mesh
/carter0/nvblox_node/mesh_marker
/carter0/nvblox_node/pointcloud
/carter0/odom
/carter0/plan
/carter0/plan_smoothed
/carter0/planner_server/transition_event
/carter0/point_cloud
/carter0/pose
/carter0/received_global_plan
/carter0/scan
/carter0/smoother_server/transition_event
/carter0/speed_limit
/carter0/tf
/carter0/tf_static
/carter0/transform
/carter0/transformed_global_plan
/carter0/velocity_smoother/transition_event
/carter0/waypoint_follower/transition_event
/clock
/diagnostics
/parameter_events
/rosout
I can't using rqt_graph(I have no idea to install it inside docker container), I'm sorry.
in more detail, It can run, but I can't see any meshes, only blank space and tf marker.
I can using nav2's 2D goal pose in that empty space, and robot moves.
Is there any specific changes need to adding namespace to nvblox node?
regards.
Lee.
A typical navgiation is to load a map using an image file and a yaml file that fills in its information.
However, your package saves point cloud data, so I wrote a question wondering how to use this data for navigation again.
Hi
Have anyone had success running the nvblox with zed on Jetson orin?
which topics from zed need to be subscribed to? what exactly are the color topics?
Thanks
AV
Hello, I am using the package you made well. I have a question during use, what does that purple area mean? I know the others are costmaps. But I'm not sure what the purple zone means or why it occurs.
Hello,
I tested nvblox using a lidar in my Isaac Sim simulation and it works well with a flat ground floor. However, in my use case the environment is not flat since there are some slopes. I'm trying nvblox in that new condition and the slopes are seen as obstacles.
From what I read it seems that the package needs a constant value which describes the height of the ground (min_height). If this is correct the package is not usable in situations where the ground is not flat.
Could you please tell me if I understood correctly the meaning of min_height parameter? What is the meaning of the max_height parameter? Is there a way to use this package with a non flat ground floor?
Thanks in advance.
Hi,
Trying to build this packages (after running rosdep install), yields the following errors:
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:739:3: error: ‘Lidar’ was not declared in this scope
739 | Lidar lidar(lidar_width_, lidar_height_, lidar_vertical_fov_rad_);
| ^~~~~
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:748:56: error: ‘lidar’ was not declared in this scope
748 | if (!converter_.checkLidarPointcloud(pointcloud_ptr, lidar)) {
| ^~~~~
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:758:21: error: ‘lidar’ was not declared in this scope
758 | pointcloud_ptr, lidar,
| ^~~~~
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:764:12: error: ‘class nvblox::RgbdMapper’ has no member named ‘integrateLidarDepth’; did you mean ‘integrateDepth’?
764 | mapper_->integrateLidarDepth(pointcloud_image_, T_S_C, lidar);
| ^~~~~~~~~~~~~~~~~~~
| integrateDepth
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp: In member function ‘void nvblox::NvbloxNode::clearMapOutsideOfRadius(const string&, const rclcpp::Time&)’:
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:903:54: error: ‘class nvblox::RgbdMapper’ has no member named ‘clearOutsideRadius’
903 | std::vector<Index3D> blocks_cleared = mapper_->clearOutsideRadius(
| ^~~~~~~~~~~~~~~~~~
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp: In member function ‘void nvblox::NvbloxNode::saveMap(std::shared_ptr<nvblox_msgs::srv::FilePath_Request_<std::allocator<void> > >, std::shared_ptr<nvblox_msgs::srv::FilePath_Response_<std::allocator<void> > >)’:
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:974:32: error: ‘class nvblox::RgbdMapper’ has no member named ‘saveMap’
974 | response->success = mapper_->saveMap(filename);
| ^~~~~~~
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp: In member function ‘void nvblox::NvbloxNode::loadMap(std::shared_ptr<nvblox_msgs::srv::FilePath_Request_<std::allocator<void> > >, std::shared_ptr<nvblox_msgs::srv::FilePath_Response_<std::allocator<void> > >)’:
/home/amram/development/nakai/packages/nakai-onboard/src/isaac_ros_nvblox/nvblox_ros/src/lib/nvblox_node.cpp:994:32: error: ‘class nvblox::RgbdMapper’ has no member named ‘loadMap’
994 | response->success = mapper_->loadMap(filename);
| ^~~~~~~
I see that there is a service to save the reconstructed mesh. Is there a way to load the saved mesh into rviz, given the initial pose? Out of the box?
Hi,
I am getting this CMAKE error while using "colcon build".
Starting >>> nvblox_ros
--- stderr: nvblox_ros
CMake Error at CMakeLists.txt:82 (set_target_properties):
set_target_properties called with incorrect number of arguments.
I am using tag v0.11.0-dp. I am doing this inside the docker that we get from isaac_ros_common with same tag.
Hi I'm using nvblox to produce a costmap for my robot to do a navigation
can it inputs Lidar pointcloud data and still works?
also, if I use other lidar based SLAM as a state estimator, would it works?
since the rgbd and double stereo IR camera and RGB camera altogether takes up too much bandwidth
If nvblox can inputs lidar data and so do vslam, then It might be able to just using lidar data.
Is that right or I am missing something?
thanks for any reply.
~/workspaces/isaac_ros-dev/scripts/run_dev.sh
there is no scripts folder in this giyhub repo
The line
git clone -b ros2-beta https://github.com/IntelRealSense/realsense-ros
in the README is outdated as described here: IntelRealSense/realsense-ros#2527
tldr;
ros2-beta branch has been renamed to ros2-development and is now the main branch, hence the branch suggested in the README here and here are outdated.
Hi I'm using ROS2 and realsense D435i to implement a navigation application, the setup are below:
in short, I implement this examples which would brings up the realsense node, nvblox, and VSLAM.
which runs well when all nodes in same machine.
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox/blob/main/docs/tutorial-nvblox-vslam-realsense.md
and I divided them for realsense node only running on the jetson board on my robots, nvblox and VSLAM runs in the edge server
robots and edge server are connected with ethernet cables, and I test they can ping and use the talker listener examples from ROS2 well.
Problems:
When I start realsense node at the jetson board side, I can see topics and image data in edge server.
But when I bring up the nvblox and VSLAM nodes, edge side doesn't see the image but the topic name still there, but jetson side can see the image data.
Is that I have any mistakes in launch files or network setup?
the launch files are as follows:
nvr_debug_jetson.launch.py
Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
NVIDIA CORPORATION and its licensors retain all intellectual property
and proprietary rights in and to this software, related documentation
and any modifications thereto. Any use, reproduction, disclosure or
distribution of this software and related documentation without an express
license agreement from NVIDIA CORPORATION is strictly prohibited.
import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import Node
from launch_ros.descriptions import ComposableNode
from launch_ros.actions import ComposableNodeContainer
def generate_launch_description():
realsense_config_file_path = os.path.join(
get_package_share_directory('nvblox_examples_bringup'),
'config', 'realsense.yaml'
)
realsense_node = ComposableNode(
namespace="camera",
package='realsense2_camera',
plugin='realsense2_camera::RealSenseNodeFactory',
parameters=[realsense_config_file_path],
)
realsense_splitter_node = ComposableNode(
namespace="camera",
name='realsense_splitter_node',
package='realsense_splitter',
plugin='nvblox::RealsenseSplitterNode',
parameters=[{
'input_qos': 'SENSOR_DATA',
'output_qos': 'SENSOR_DATA'
}],
remappings=[('input/infra_1', '/camera/infra1/image_rect_raw'),
('input/infra_1_metadata', '/camera/infra1/metadata'),
('input/infra_2', '/camera/infra2/image_rect_raw'),
('input/infra_2_metadata', '/camera/infra2/metadata'),
('input/depth', '/camera/depth/image_rect_raw'),
('input/depth_metadata', '/camera/depth/metadata'),
('input/pointcloud', '/camera/depth/color/points'),
('input/pointcloud_metadata', '/camera/depth/metadata'),
]
)
realsense_container = ComposableNodeContainer(
name='realsense_container',
namespace='',
package='rclcpp_components',
executable='component_container',
composable_node_descriptions=[
realsense_node,
realsense_splitter_node
],
output='screen'
)
base_link_tf_node = Node(
package='tf2_ros',
executable='static_transform_publisher',
arguments=[
'0.16', '0', '0.11', '0', '0', '0', '1',
'base_link', 'camera_link']
)
return LaunchDescription([
realsense_container,
base_link_tf_node
])
nvr_debug_edge.launch.py
Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
NVIDIA CORPORATION and its licensors retain all intellectual property
and proprietary rights in and to this software, related documentation
and any modifications thereto. Any use, reproduction, disclosure or
distribution of this software and related documentation without an express
license agreement from NVIDIA CORPORATION is strictly prohibited.
import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import Node
from launch_ros.descriptions import ComposableNode
from launch_ros.actions import ComposableNodeContainer
def generate_launch_description():
visual_slam_node = Node(
name='visual_slam_node',
package='isaac_ros_visual_slam',
executable='isaac_ros_visual_slam',
parameters=[{
'enable_rectified_pose': True,
'denoise_input_images': False,
'rectified_images': True,
'enable_debug_mode': False,
'debug_dump_path': '/tmp/vslam',
'enable_slam_visualization': True,
'enable_landmarks_view': True,
'enable_observations_view': True,
'map_frame': 'map',
'odom_frame': 'odom',
'base_frame': 'base_link',
'input_left_camera_frame': 'camera_infra1_frame',
'input_right_camera_frame': 'camera_infra2_frame',
'enable_localization_n_mapping': True,
'publish_odom_to_base_tf': True,
'publish_map_to_odom_tf': True,
}],
remappings=[('stereo_camera/left/image', '/camera/infra1/image_rect_raw'),
('stereo_camera/left/camera_info', '/camera/infra1/camera_info'),
('stereo_camera/right/image', '/camera/infra2/image_rect_raw'),
('stereo_camera/right/camera_info', '/camera/infra2/camera_info')]
)
nvblox_config = DeclareLaunchArgument(
'nvblox_config', default_value=os.path.join(
get_package_share_directory(
'nvblox_examples_bringup'), 'config', 'nvblox.yaml'
)
)
nvblox_node = Node(
package='nvblox_ros',
executable='nvblox_node',
parameters=[LaunchConfiguration('nvblox_config')],
output='screen',
remappings=[
("depth/camera_info", "/camera/depth/camera_info"),
("depth/image", "/camera/realsense_splitter_node/output/depth"),
("color/camera_info", "/camera/color/camera_info"),
("color/image", "/camera/color/image_raw")
]
)
rviz_config_path = os.path.join(get_package_share_directory(
'nvblox_examples_bringup'), 'config', 'nvblox_vslam_realsense.rviz')
print(rviz_config_path)
rviz = Node(
package='rviz2',
executable='rviz2',
arguments=['-d', rviz_config_path],
output='screen')
return LaunchDescription([
nvblox_config,
visual_slam_node,
nvblox_node,
rviz
])
Hello Nvidia,
thanks a lot for your awesome work on Nvblox.
It would be great if you could also provide an official ROS1 wrapper for this package.
Most deployed robots in research are still operating on ROS1.
This would be a great enhancement and benefit a large part of the community.
Best Jonas
Hi Team,
I am facing a cmake error while colcon building the packages inside the docker container. Any idea about the issue?
CMake Error at CMakeLists.txt:49 (find_package):
By not providing "Findnvblox.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "nvblox", but
CMake did not find one.
Could not find a package configuration file provided by "nvblox" with any
of the following names:
nvbloxConfig.cmake
nvblox-config.cmake
Add the installation prefix of "nvblox" to CMAKE_PREFIX_PATH or set
"nvblox_DIR" to a directory containing one of the above files. If "nvblox"
provides a separate development package or SDK, be sure it has been
installed.
Regards,
Rafi
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox/blob/main/docs/tutorial-human-reconstruction-realsense.md
Hello, I ran into this problem when executing this example.
Starting >>> isaac_ros_stereo_image_proc
Finished <<< isaac_ros_nitros_detection2_d_array_type [26.0s]
--- stderr: isaac_ros_image_proc
CMake Error at gxf/image_flip/CMakeLists.txt:21 (include):
include could not find requested file:
YamlCpp
CMake Error at gxf/image_flip/CMakeLists.txt:22 (find_package):
No "FindGXF.cmake" found in CMAKE_MODULE_PATH.
CMake Warning (dev) at gxf/image_flip/CMakeLists.txt:22 (find_package):
FindGXF.cmake must either be part of this project itself, in this case
adjust CMAKE_MODULE_PATH so that it points to the correct location inside
its source tree.
Or it must be installed by a package which has already been found via
find_package(). In this case make sure that package has indeed been found
and adjust CMAKE_MODULE_PATH to contain the location where that package has
installed FindGXF.cmake. This must be a location provided by that package.
This error in general means that the buildsystem of this project is relying
on a Find-module without ensuring that it is actually available.
This warning is for project developers. Use -Wno-dev to suppress it.
Failed <<< isaac_ros_image_proc [2.16s, exited with code 1]
Aborted <<< isaac_ros_stereo_image_proc [2.80s]
Aborted <<< isaac_ros_visual_slam [27.6s]
Aborted <<< nvblox [4min 15s]
So I'm just trying to get nvblox working with realsense so I followed the steps here, twice.
There are two things that happen,
cd /tmp && \
git clone https://github.com/IntelRealSense/librealsense && \
cd librealsense && \
./scripts/setup_udev_rules.sh
from here, after entering my password I get a lot of permissions denied. So I ran the script using sudo and it works this time.
rosdep installs inside the container from the 5th step from here I get this:$ rosdep install -i -r --from-paths src --rosdistro humble -y --skip-keys "libopencv-dev libopencv-contrib-dev libopencv-imgproc-dev python-opencv python3-opencv nvblox"
ERROR: the following packages/stacks could not have their rosdep keys resolved
to system dependencies:
realsense2_camera: No definition of [librealsense2] for OS version [focal]
realsense2_description: No definition of [xacro] for OS version [focal]
Continuing to install resolvable dependencies...
#All required rosdeps installed successfully
Is isaac_ros_nvblox suitable for acceleration speed when used with 3d lidar? How many times does it speed up? Thanks
I use the command pip3 install nvidia-pyindex and pip3 install nvidia-tao to install it ,but in my computer didn't appear /opt/nvidia/tao/tao-converter. Can you explain this step in detail.
docker: Error response from daemon: failed to create shim: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running hook #0: error running hook: exit status 1, stdout: src: /etc/vulkan/icd.d/nvidia_icd.json, src_lnk: /usr/lib/aarch64-linux-gnu/tegra/nvidia_icd.json, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/etc/vulkan/icd.d/nvidia_icd.json, dst_lnk: /usr/lib/aarch64-linux-gnu/tegra/nvidia_icd.json
src: /usr/lib/aarch64-linux-gnu/libcuda.so, src_lnk: tegra/libcuda.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libcuda.so, dst_lnk: tegra/libcuda.so
src: /usr/lib/aarch64-linux-gnu/libnvcucompat.so, src_lnk: tegra/libnvcucompat.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libnvcucompat.so, dst_lnk: tegra/libnvcucompat.so
src: /usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so, src_lnk: tegra/libnvidia-nvvm.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libnvidia-nvvm.so, dst_lnk: tegra/libnvidia-nvvm.so
src: /usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999, src_lnk: tegra/libnvv4l2.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libv4l2.so.0.0.999999, dst_lnk: tegra/libnvv4l2.so
src: /usr/lib/aarch64-linux-gnu/libv4lconvert.so.0.0.999999, src_lnk: tegra/libnvv4lconvert.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libv4lconvert.so.0.0.999999, dst_lnk: tegra/libnvv4lconvert.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvargus.so, src_lnk: ../../../tegra/libv4l2_nvargus.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvargus.so, dst_lnk: ../../../tegra/libv4l2_nvargus.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvcuvidvideocodec.so, src_lnk: ../../../tegra/libv4l2_nvcuvidvideocodec.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvcuvidvideocodec.so, dst_lnk: ../../../tegra/libv4l2_nvcuvidvideocodec.so
src: /usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvvideocodec.so, src_lnk: ../../../tegra/libv4l2_nvvideocodec.so, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libv4l/plugins/nv/libv4l2_nvvideocodec.so, dst_lnk: ../../../tegra/libv4l2_nvvideocodec.so
src: /usr/lib/aarch64-linux-gnu/libvulkan.so.1.3.203, src_lnk: tegra/libvulkan.so.1.3.203, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/libvulkan.so.1.3.203, dst_lnk: tegra/libvulkan.so.1.3.203
src: /usr/lib/aarch64-linux-gnu/tegra/libcuda.so, src_lnk: libcuda.so.1.1, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libcuda.so, dst_lnk: libcuda.so.1.1
src: /usr/lib/aarch64-linux-gnu/tegra/libgstnvdsseimeta.so, src_lnk: libgstnvdsseimeta.so.1.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libgstnvdsseimeta.so, dst_lnk: libgstnvdsseimeta.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so, src_lnk: libnvbufsurface.so.1.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbufsurface.so, dst_lnk: libnvbufsurface.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so, src_lnk: libnvbufsurftransform.so.1.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbufsurftransform.so, dst_lnk: libnvbufsurftransform.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so, src_lnk: libnvbuf_utils.so.1.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvbuf_utils.so, dst_lnk: libnvbuf_utils.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvdsbufferpool.so, src_lnk: libnvdsbufferpool.so.1.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvdsbufferpool.so, dst_lnk: libnvdsbufferpool.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so, src_lnk: libnvidia-nvvm.so.4, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so, dst_lnk: libnvidia-nvvm.so.4
src: /usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so.4, src_lnk: libnvidia-nvvm.so.4.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvidia-nvvm.so.4, dst_lnk: libnvidia-nvvm.so.4.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so, src_lnk: libnvid_mapper.so.1.0.0, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvid_mapper.so, dst_lnk: libnvid_mapper.so.1.0.0
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscibuf.so, src_lnk: libnvscibuf.so.1, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscibuf.so, dst_lnk: libnvscibuf.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscicommon.so, src_lnk: libnvscicommon.so.1, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscicommon.so, dst_lnk: libnvscicommon.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscistream.so, src_lnk: libnvscistream.so.1, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscistream.so, dst_lnk: libnvscistream.so.1
src: /usr/lib/aarch64-linux-gnu/tegra/libnvscisync.so, src_lnk: libnvscisync.so.1, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/lib/aarch64-linux-gnu/tegra/libnvscisync.so, dst_lnk: libnvscisync.so.1
src: /usr/share/glvnd/egl_vendor.d/10_nvidia.json, src_lnk: ../../../lib/aarch64-linux-gnu/tegra-egl/nvidia.json, dst: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/usr/share/glvnd/egl_vendor.d/10_nvidia.json, dst_lnk: ../../../lib/aarch64-linux-gnu/tegra-egl/nvidia.json
, stderr: nvidia-container-cli: mount error: file creation failed: /var/lib/docker/overlay2/0aba4fc4532ba2ddadb7949384e51a82af51ed554581ce0440af85e3805d50aa/merged/dev/nvhost-as-gpu: invalid argument: unknown.
~/workspaces/isaac_ros-dev/src/isaac_ros_common
Required more information about using the nvblox for dynamic obstacles, in the particular clearing. The attached video shows the nvblox running on a desktop computer. The costmap shows the adding and clearing of the obstacles.
Both the ESDF and the TSDF are updated at 30 FPS. Thanks in advance for your help.
First of all, I'm sorry it's a simple question, not an issue related.
There seems to be a part in tsdf that uses visual odometry in the middle to read your thesis and find related materials.
Can I use that part for navigation?
Hi,
Can someone explain to me the hardcoded value in the carter_sim.launch.py? Why this offset is needed?
Hi, I’m modifying the carter_sim_elbrus.launch.py example in nvblox_nav2 to do navigation in my robot,
The application is ok at DP1.1 release before
but for DP2.0, after the configurinig is done, the terminal keep popping “waiting for server state”
Error Output:
[lifecycle_manager-10] [INFO] [1666756876.922715146] [lifecycle_manager_navigation]: Waiting for service controller_server/get_state...
[lifecycle_manager-1] [INFO] [1666756876.922716862] [lifecycle_manager_map]: Waiting for service map_server/get_state...
[lifecycle_manager-1] [INFO] [1666756878.923016306] [lifecycle_manager_map]: Waiting for service map_server/get_state...
[lifecycle_manager-10] [INFO] [1666756878.923042882] [lifecycle_manager_navigation]: Waiting for service controller_server/get_state...
[lifecycle_manager-1] [INFO] [1666756880.923268323] [lifecycle_manager_map]: Waiting for service map_server/get_state...
[lifecycle_manager-10] [INFO] [1666756880.923292501] [lifecycle_manager_navigation]: Waiting for service controller_server/get_state...
[lifecycle_manager-10] [INFO] [1666756882.923515151] [lifecycle_manager_navigation]: Waiting for service controller_server/get_state...
[lifecycle_manager-1] [INFO] [1666756882.923517381] [lifecycle_manager_map]: Waiting for service map_server/get_state...
[lifecycle_manager-1] [INFO] [1666756884.923782959] [lifecycle_manager_map]: Waiting for service map_server/get_state...
[lifecycle_manager-10] [INFO] [1666756884.923803742] [lifecycle_manager_navigation]: Waiting for service controller_server/get_state...
[nvblox_node-11] [INFO] [1666756885.119232012] [nvblox_node]: RGB frame statistics:
[nvblox_node-11] avg=66.713883, min=55.955423, max=84.688353, std_dev=5.970719, count=13211
[nvblox_node-11] [INFO] [1666756885.463369156] [nvblox_node]: Depth frame statistics:
[nvblox_node-11] avg=36.677327, min=13.215166, max=126.695587, std_dev=10.915946, count=24039
[nvblox_node-11] [INFO] [1666756885.463661005] [nvblox_node]: Timing statistics:
[nvblox_node-11] NVBlox Timing
[nvblox_node-11] -----------
[nvblox_node-11] color/integrate
But the nvblox and vslam seem working as expect
so maybe is that the navigation stack doesn’t working since there are no availiable map input even though the nvblox does produced the map, so I assume that the nvblox has output esdf map but map server didn’t receive it
My launch files:
carter_sim_elbrus_new.launch.py
(for this the I didn't modified the map server related lines)
import os
from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.actions import IncludeLaunchDescription
from launch.conditions import IfCondition
from launch.launch_description_sources import PythonLaunchDescriptionSource
from launch.substitutions import LaunchConfiguration
from launch_ros.actions import ComposableNodeContainer
from launch_ros.actions import Node
from launch_ros.descriptions import ComposableNode
def generate_launch_description():
#simulation time, this would not be use in the our application
use_sim_time = LaunchConfiguration('use_sim_time', default='False')
#costmap param, need to be modify
param_dir = LaunchConfiguration(
'params_file',
default=os.path.join(
get_package_share_directory(
'nvblox_nav2'), 'params', 'carter_nav2_new.yaml'
),
)
#nvblox params, if the output is not good, tune later
nvblox_param_dir = LaunchConfiguration(
'nvblox_params_file', default=os.path.join(
get_package_share_directory('nvblox_nav2'),
'params', 'nvblox.yaml'),)
nav2_bringup_launch_dir = os.path.join(
get_package_share_directory('nav2_bringup'), 'launch')
rviz_config_dir = os.path.join(get_package_share_directory(
'nvblox_nav2'), 'config', 'carter_nvblox_nav2.rviz')
lifecycle_nodes = ['map_server']
#setting up vslam node, to be modify, we use infrared image here
visual_slam_node = ComposableNode(
name='visual_slam_node',
package='isaac_ros_visual_slam',
plugin='isaac_ros::visual_slam::VisualSlamNode',
remappings=[('/stereo_camera/left/camera_info', '/camera/infra1/camera_info'),
('/stereo_camera/right/camera_info', '/camera/infra2/camera_info'),
('/stereo_camera/left/image', '/camera/realsense_splitter_node/output/infra_1'),
('/stereo_camera/right/image', '/camera/realsense_splitter_node/output/infra_2')],
parameters=[{
'image_qos': 'SENSOR_DATA',
'enable_rectified_pose': True,
'denoise_input_images': True,
'rectified_images': True,
'enable_debug_mode': False,
'debug_dump_path': '/tmp/elbrus',
'left_camera_frame': 'camera_infra1_frame',
'right_camera_frame': 'camera_infra2_frame',
'map_frame': 'map',
'fixed_frame': 'odom',
'odom_frame': 'odom',
'base_frame': 'base_link',
#these two does not have reference
'current_smooth_frame': 'base_link_smooth',
'current_rectified_frame': 'base_link_rectified',
'enable_observations_view': True,
'enable_landmarks_view': True,
'enable_reading_slam_internals': True,
'enable_slam_visualization': True,
'enable_localization_n_mapping': True,
'use_sim_time': use_sim_time
}]
)
visual_slam_launch_container = ComposableNodeContainer(
name='visual_slam_launch_container',
namespace='',
package='rclcpp_components',
executable='component_container',
composable_node_descriptions=[
visual_slam_node
],
output='screen'
)
#now vslam should be set
return LaunchDescription(
[
#this lacks the realsense node, would add it in later(on the above)
#here we don't run the realsense at edge
#realsense_container,
#adding TF joints to the camera and baselink
#base_link_tf_node,
#loading args of params file
DeclareLaunchArgument(
'params_file', default_value=param_dir,
description='Full path to param file to load'),
#is it simulation, in our application it is not, it is false
DeclareLaunchArgument(
'use_sim_time', default_value='False',
description='Use simulation (Omniverse Isaac Sim) clock if true'),
#rviz args, leave it
DeclareLaunchArgument(
'run_rviz', default_value='true',
description='Whether to start RVIZ'),
#running map_server
DeclareLaunchArgument(
'run_nav2', default_value='true',
description='Whether to run nav2'),
Node(
package='nav2_lifecycle_manager', executable='lifecycle_manager',
name='lifecycle_manager_map', output='screen',
parameters=[{'use_sim_time': use_sim_time},
{'autostart': True},
{'node_names': lifecycle_nodes}],
condition=IfCondition(
LaunchConfiguration('run_nav2'))
),
#bringup rviz
IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(nav2_bringup_launch_dir, 'rviz_launch.py')),
launch_arguments={'namespace': '', 'use_namespace': 'False',
'autostart': 'True',
'rviz_config': rviz_config_dir}.items(),
condition=IfCondition(LaunchConfiguration('run_rviz')),),
#setting up nav2 stack and bring up, points to carter_nav2_new.yaml
IncludeLaunchDescription(
PythonLaunchDescriptionSource(
os.path.join(
nav2_bringup_launch_dir, 'navigation_launch.py')),
launch_arguments={'params_file': param_dir,
'use_sim_time': use_sim_time,
'autostart': 'True'}.items(),
condition=IfCondition(
LaunchConfiguration('run_nav2'))
),
#bringing up nvblox, modify the camera input
Node(
package='nvblox_ros', executable='nvblox_node',
parameters=[nvblox_param_dir, {'use_sim_time': use_sim_time,
'global_frame': 'odom'}],
output='screen',
remappings=[('depth/image', '/camera/realsense_splitter_node/output/depth'),
('depth/camera_info', '/camera/depth/camera_info'),
('color/image', '/camera/color/image_raw'),
('color/camera_info', '/camera/color/camera_info'), ]),
#bringing up vslam
visual_slam_launch_container,
])
carter_nav2_new.yaml
(for this I only modified the tolerance and threshold to fit my own robot, the map form or name didn't changed)
bt_navigator:
ros__parameters:
use_sim_time: True
global_frame: map
robot_base_frame: base_link
odom_topic: /odom
bt_loop_duration: 20
default_server_timeout: 40
# 'default_nav_through_poses_bt_xml' and 'default_nav_to_pose_bt_xml' are use defaults:
# nav2_bt_navigator/navigate_to_pose_w_replanning_and_recovery.xml
# nav2_bt_navigator/navigate_through_poses_w_replanning_and_recovery.xml
# They can be set here or via a RewrittenYaml remap from a parent launch file to Nav2.
plugin_lib_names:
- nav2_compute_path_to_pose_action_bt_node
- nav2_compute_path_through_poses_action_bt_node
- nav2_smooth_path_action_bt_node
- nav2_follow_path_action_bt_node
- nav2_spin_action_bt_node
- nav2_wait_action_bt_node
- nav2_back_up_action_bt_node
- nav2_drive_on_heading_bt_node
- nav2_clear_costmap_service_bt_node
- nav2_is_stuck_condition_bt_node
- nav2_goal_reached_condition_bt_node
- nav2_goal_updated_condition_bt_node
- nav2_globally_updated_goal_condition_bt_node
- nav2_is_path_valid_condition_bt_node
- nav2_initial_pose_received_condition_bt_node
- nav2_reinitialize_global_localization_service_bt_node
- nav2_rate_controller_bt_node
- nav2_distance_controller_bt_node
- nav2_speed_controller_bt_node
- nav2_truncate_path_action_bt_node
- nav2_truncate_path_local_action_bt_node
- nav2_goal_updater_node_bt_node
- nav2_recovery_node_bt_node
- nav2_pipeline_sequence_bt_node
- nav2_round_robin_node_bt_node
- nav2_transform_available_condition_bt_node
- nav2_time_expired_condition_bt_node
- nav2_path_expiring_timer_condition
- nav2_distance_traveled_condition_bt_node
- nav2_single_trigger_bt_node
- nav2_is_battery_low_condition_bt_node
- nav2_navigate_through_poses_action_bt_node
- nav2_navigate_to_pose_action_bt_node
- nav2_remove_passed_goals_action_bt_node
- nav2_planner_selector_bt_node
- nav2_controller_selector_bt_node
- nav2_goal_checker_selector_bt_node
- nav2_controller_cancel_bt_node
- nav2_path_longer_on_approach_bt_node
- nav2_wait_cancel_bt_node
- nav2_spin_cancel_bt_node
- nav2_back_up_cancel_bt_node
- nav2_drive_on_heading_cancel_bt_node
bt_navigator_rclcpp_node:
ros__parameters:
use_sim_time: True
controller_server:
ros__parameters:
use_sim_time: True
controller_frequency: 10.0
min_x_velocity_threshold: 0.001
min_y_velocity_threshold: 0.05
min_theta_velocity_threshold: 0.001
#change the failure_tolerance for tunning
failure_tolerance: 2
progress_checker_plugin: "progress_checker"
goal_checker_plugin: ["general_goal_checker"]
controller_plugins: ["FollowPath"]
# Progress checker parameters
progress_checker:
plugin: "nav2_controller::SimpleProgressChecker"
required_movement_radius: 0.5
movement_time_allowance: 10.0
# Goal checker parameters
general_goal_checker:
plugin: "nav2_controller::SimpleGoalChecker"
#changed goal tolerance for tunning
xy_goal_tolerance: 0.4
yaw_goal_tolerance: 0.4
stateful: True
# DWB parameters
FollowPath:
plugin: "dwb_core::DWBLocalPlanner"
debug_trajectory_details: True
min_vel_x: 0.0
min_vel_y: 0.0
max_vel_x: 0.2
max_vel_y: 0.0
max_vel_theta: 0.2
min_speed_xy: 0.0
max_speed_xy: 0.2
min_speed_theta: 0.0
# Add high threshold velocity for turtlebot 3 issue.
# https://github.com/ROBOTIS-GIT/turtlebot3_simulations/issues/75
acc_lim_x: 2.5
acc_lim_y: 0.0
acc_lim_theta: 1.2
decel_lim_x: -2.5
decel_lim_y: 0.0
decel_lim_theta: -1.2
vx_samples: 30
vy_samples: 10
vtheta_samples: 20
sim_time: 1.7
linear_granularity: 0.05
angular_granularity: 0.025
transform_tolerance: 0.2
xy_goal_tolerance: 0.25
trans_stopped_velocity: 0.25
short_circuit_trajectory_evaluation: True
stateful: True
critics: ["RotateToGoal", "Oscillation", "BaseObstacle", "GoalAlign", "PathAlign", "PathDist", "GoalDist"]
BaseObstacle.scale: 0.02
PathAlign.scale: 32.0
PathAlign.forward_point_distance: 0.1
GoalAlign.scale: 24.0
GoalAlign.forward_point_distance: 0.1
PathDist.scale: 32.0
GoalDist.scale: 24.0
RotateToGoal.scale: 32.0
RotateToGoal.slowing_factor: 1.0
RotateToGoal.lookahead_time: -1.0
controller_server_rclcpp_node:
ros__parameters:
use_sim_time: True
local_costmap:
local_costmap:
ros__parameters:
update_frequency: 10.0
publish_frequency: 10.0
global_frame: map
robot_base_frame: base_link
use_sim_time: True
rolling_window: True
width: 10
height: 10
resolution: 0.05
robot_radius: 0.4
plugins: ["nvblox_layer"]
nvblox_layer:
plugin: "nvblox::nav2::NvbloxCostmapLayer"
enabled: True
max_obstacle_distance: 1.0
inflation_distance: 0.4
always_send_full_costmap: True
local_costmap_client:
ros__parameters:
use_sim_time: True
local_costmap_rclcpp_node:
ros__parameters:
use_sim_time: True
global_costmap:
global_costmap:
ros__parameters:
update_frequency: 10.0
publish_frequency: 10.0
global_frame: map
robot_base_frame: base_link
use_sim_time: True
rolling_window: True
width: 200
height: 200
robot_radius: 0.4
resolution: 0.1
origin_x: -100.0
origin_y: -100.0
plugins: ["nvblox_layer"]
nvblox_layer:
plugin: "nvblox::nav2::NvbloxCostmapLayer"
enabled: True
max_obstacle_distance: 1.0
inflation_distance: 0.4
always_send_full_costmap: True
global_costmap_client:
ros__parameters:
use_sim_time: True
global_costmap_rclcpp_node:
ros__parameters:
use_sim_time: True
planner_server:
ros__parameters:
expected_planner_frequency: 10.0
use_sim_time: True
planner_plugins: ["GridBased"]
GridBased:
plugin: "nav2_navfn_planner/NavfnPlanner"
tolerance: 0.5
use_astar: false
allow_unknown: true
planner_server_rclcpp_node:
ros__parameters:
use_sim_time: True
smoother_server:
ros__parameters:
use_sim_time: True
smoother_plugins: ["simple_smoother"]
simple_smoother:
plugin: "nav2_smoother::SimpleSmoother"
tolerance: 1.0e-10
max_its: 1000
do_refinement: True
behavior_server:
ros__parameters:
costmap_topic: local_costmap/costmap_raw
footprint_topic: local_costmap/published_footprint
cycle_frequency: 5.0
behavior_plugins: ["spin", "backup", "drive_on_heading", "wait"]
spin:
plugin: "nav2_behaviors/Spin"
backup:
plugin: "nav2_behaviors/BackUp"
drive_on_heading:
plugin: "nav2_behaviors/DriveOnHeading"
wait:
plugin: "nav2_behaviors/Wait"
global_frame: map
robot_base_frame: base_link
transform_tolerance: 0.2
use_sim_time: true
simulate_ahead_time: 2.0
max_rotational_vel: 0.2
min_rotational_vel: 0.05
rotational_acc_lim: 3.2
robot_state_publisher:
ros__parameters:
use_sim_time: True
waypoint_follower:
ros__parameters:
loop_rate: 20
stop_on_failure: false
waypoint_task_executor_plugin: "wait_at_waypoint"
wait_at_waypoint:
plugin: "nav2_waypoint_follower::WaitAtWaypoint"
enabled: True
waypoint_pause_duration: 200
has anyone ran into this problem? and any advises are appreciated.
Thanks
hi,
I’m using velodyne vlp16 lidar as nvblox input
at first it shows up correct mesh even if I move the robot for few meters
but after I move the robot further and steady, the robot in just teleported to a different area,
and then the mesh just mess up,
and terminal has output
[nvblox_node-10] WARNING! Mesh block is too big to weld. Exiting without doing anything.
[nvblox_node-10] WARNING! Mesh block is too big to weld. Exiting without doing anything.
[nvblox_node-10] WARNING! Mesh block is too big to weld. Exiting without doing anything.
I thought it might be the allocated block are not enough?
I have tuned the voxel_size larger to 0.1 so as to reduce computation, but it seems no use.
so how can I tune for it?
thanks for any assistance.
nvblox_lidar.yaml:
nvblox_node:
ros__parameters:
global_frame: odom
#voxel size changed from 0.05 to 0.1 to lower the computation
voxel_size: 0.1
slice_height: 0.75
#min_height: -0.25
#min_height: -0.1
min_height: 0.1
max_height: 1.0
esdf: true
esdf_2d: true
mesh: true
distance_slice: true
use_color: true
#lidar parameters:
use_depth: false
use_lidar: true
lidar_width: 1800
lidar_height: 16
lidar_vertical_fov_rad: 0.5235987756
tsdf_integrator_max_integration_distance_m: 4.0
lidar_tsdf_integrator_max_integration_distance_m: 10.0
tsdf_integrator_truncation_distance_vox: 4.0
tsdf_integrator_max_weight: 20.0
esdf_integrator_min_weight: 2.0
esdf_integrator_max_distance_m: 2.0
esdf_integrator_max_site_distance_vox: 5.0
max_tsdf_update_hz: 0.0
max_pointcloud_update_hz: 0.0
max_color_update_hz: 5.0
max_mesh_update_hz: 1.0
max_esdf_update_hz: 10.0
#lidar_width: 1800
#lidar_height: 31
output_dir: /tmp/
Hi Team,
We are working with Jetson Orin and v0.11.0-dp Isaac ROS GEMS packages.
As per the following image shown, nvblox and vslam is linked via /pose topic (or data).
However if I checked ros2 node info /nvblox_node and
ros2 node info /visual_slam_node,
it looks like NVBLOX node is subscribed to /pose topic, however VSLAM node is not publishing /pose topic. Also, we can not able to see REMAPPINGS of topics in any of the launch file given in https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox/tree/v0.11.0-dp/nvblox_nav2/launch or https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox/tree/v0.11.0-dp/nvblox_examples/nvblox_examples_bringup/launch
How does than nvblox consuming data published by vslam node?
Hello!
I'm using Rviz2 to visualize the distance map slice and I sometimes see a disagreement between the origin of the distance map slice and the point cloud slice. It seems that the origin of the distance map slice sometimes is off by one voxel in either (or both) x and y directions. When i move the robot around I can see the offset shifting around once in a while.
Note that i convert the nvblox distance map slice to a nav_msgs::msg::OccupancyGrid msg to enable visualization in Rviz, but their msg fields should have a 1-1 correspondence.
I think this could be a potential issue in the implementation of: RosConverter::distanceMapSliceFromLayer(), but I'm not sure.
Hi @helenol,
From your experience using the Intel realsense cameras in indoor environments, just wondering if you may be able to help elaborate on the following rgbd mapper parameters, which I am trying to tune to get the best possible results using the D455/D435 cameras to compare with the Sun3D dataset.
Parameter | value | Remarks
In addition to the integrator parameter, I have used the following:
Where possible, I have looked into the source code and the readme files but another look or more explanation would be greatly appreciated. The example provided using the Sun3D dataset using Asus Xiton Pro seems to produce better results than the realsense and I am not sure why this is the case. Note that the depth image encoding is 16UC1 from the realsense sensor and therefore the converter performs the division. Thanks in advance.
CUDA error = 2 at /home/nvidia/zed_ws/src/isaac_ros_nvblox/nvblox/nvblox/include/nvblox/core/impl/unified_vector_impl.h:209 'cudaMalloc(&new_buffer, sizeof(T) * capacity)'. Error string: out of memory.
any solution?
Hi,
I'm currently trying to run nvblox on my x86 system with a realsense.
As with some other issues here, I'm facing the following error Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
I followed this guide in order to install librealsense2-dkms.
Unfortunately, this guide does not work for me. My system runs on kernel version 5.19.0-41-generic. The linked download at the guide only supports 5.13/5.15.
As far as I see it, there is no librealsense2-dkms package for kernel version 5.19.
Is there a possible solution to solve this issue without downgrading my kernel?
Many thanks in advance!
This is more like a question than an Issue. I just wanted to know if nvblox algorithm has been tested on AGX or any Xavier with realsense or any other camera to recreate a 3D environment IRL(in real life)? Or is it tested in Simulation only? Why I am asking is because when I try to recreate a 3D environment, the re-construction is not much accurate (in comparison to the Isaac Sim result.) Also, if you have tested it IRL then I will try and play with the different parameters to get better output.
I'm unable to build nvblox from source on Jetson Xavier NX.
Built ROS2 foxy from source, Built Navigation2 packages, Realsense packages and Isaac Elbrus Slam ROS packages. Everything works without a problem.
Sourced ROS2, While trying to build nvblox, nvblox_ros is unable to locate tf2_eigen.h header file. Although, the dependency in CMakelists.txt is met and can also verify that the packages are found under ~/ros2_foxy/. Any thoughts on the issue?
`--- stderr: nvblox_ros
CMake Warning (dev) in CMakeLists.txt:
Policy CMP0104 is not set: CMAKE_CUDA_ARCHITECTURES now detected for NVCC,
empty CUDA_ARCHITECTURES not allowed. Run "cmake --help-policy CMP0104"
for policy details. Use the cmake_policy command to set the policy and
suppress this warning.
CUDA_ARCHITECTURES is empty for target "nvblox_ros_lib".
This warning is for project developers. Use -Wno-dev to suppress it.
In file included from /home/eloopai/nav2_ws/src/isaac_ros_nvblox/nvblox_ros/src/transformer.cpp:11:0:
/home/eloopai/nav2_ws/src/isaac_ros_nvblox/nvblox_ros/include/nvblox_ros/transformer.hpp:15:10: fatal error: tf2_eigen/tf2_eigen.h: No such file or directory
#include <tf2_eigen/tf2_eigen.h>
^~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[2]: *** [CMakeFiles/nvblox_ros_lib.dir/src/transformer.cpp.o] Error 1
make[1]: *** [CMakeFiles/nvblox_ros_lib.dir/all] Error 2
make: *** [all] Error 2
---`
Hello,
We can succeeded to run below demo.
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox/blob/main/docs/tutorial-human-reconstruction-isaac-sim.md
But Robot(Carter) sometimes go into narrow area.
Then the Carter can't move like below;
Question:
Are there any recommended way to solve this problem?
--
Best regards,
Hello,
Is there any support to be able to use the nvblox capabilities from Isaac ROS nvbox in parallel environments in Omniverse Sim/Gym? The ROS2 layer would have to be stripped away as it's not appropriate for the parallel Gym environments used during the RL training. Is this possible?
Hi, I was trying to explore the nvblox and wondering if it can support multiple depth cameras? In case it is not supported, please guide us on how to contribute and some pointers which will help to add this support. Thanks.
I am running isaac_ros_nvblox package inside docker container. I followed the ROS docker setup steps as per the README file.
When I tried installing ROS workspace dependencies with rosdep install -i -r --from-paths src --rosdistro foxy -y --skip-keys "libopencv-dev libopencv-contrib-dev libopencv-imgproc-dev python-opencv python3-opencv", I got following error:
ERROR: the following packages/stacks could not have their rosdep keys resolved to system dependencies: realsense2_camera: No definition of [librealsense2] for OS version [bionic] realsense2_description: No definition of [xacro] for OS version [bionic] nvblox_ros: Cannot locate rosdep definition for [nvblox] Continuing to install resolvable dependencies... #All required rosdeps installed successfully
When I used ros2 run nvblox_ros nvblox_node, I got this error error while loading shared libraries: libnvblox_lib.so: cannot open shared object file: No such file or directory.
How can I run the nvblox_node? Can anyone help? Thanks.
I don't know if this feature already exists within the core library, but is there a method to prune 'far away' voxels or map blocks? :)
Hello, I have been trying to follow the Tutorial For Realsense-based Reconstruction on a Jetson Xavier NX 16GB board running a freshly flashed Jetpack 5.1.1 install. As suggested by the tutorial, all components are set up in the dev Docker container on an external SSD and all dependencies are able to be resolved by rosdep.
However, when I try to build my ros 2 workspace using colcon, I get an error indicating that CMake can not find nvToolsExt. When running the Docker container and nvblox environment as shown in the tutorial, I get the following error:
Starting >>> nvblox
--- stderr: nvblox
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
CUDA_nvToolsExt_LIBRARY (ADVANCED)
linked by target "nvblox_lib" in directory /workspaces/isaac_ros-dev/src/isaac_ros_nvblox/nvblox/nvblox
linked by target "nvblox_gpu_hash" in directory /workspaces/isaac_ros-dev/src/isaac_ros_nvblox/nvblox/nvblox
CMake Generate step failed. Build files cannot be regenerated correctly.
---
Failed <<< nvblox [8.88s, exited with code 1]
Aborted <<< isaac_ros_nitros [33.5s]I have verified that cuda-nvtx-11-4 is installed on the host and that the shared objects relating to nvToolsExt as well as nvToolsExtCuda.h are present. Furthemore, I did try to alter the run_dev.sh script to mount libnvToolsExt.so.1.0.0 and libnvToolsExt.so.1 to the container in addition to libnvToolsExt.so, which is mounted by default.
As a last resort, I have tried to manually set CUDA_nvToolsExt_LIBRARY statically in nvblox' CmakeLists.txt to point to the correct libraries. While the build progresses further than before, the linker produces the following error, saying that it can not link to nvToolsExt:
Starting >>> nvblox
[Processing: isaac_ros_nitros, nvblox]
--- stderr: isaac_ros_nitros
/usr/bin/ld: cannot find -lnvToolsExt
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/isaac_ros_nitros_exe.dir/build.make:162: isaac_ros_nitros_exe] Error 1
make[1]: *** [CMakeFiles/Makefile2:185: CMakeFiles/isaac_ros_nitros_exe.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
/usr/bin/ld: cannot find -lnvToolsExt
/usr/bin/ld: cannot find -lnvToolsExt
collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/isaac_ros_nitros.dir/build.make:359: libisaac_ros_nitros.so] Error 1
make[1]: *** [CMakeFiles/Makefile2:159: CMakeFiles/isaac_ros_nitros.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
---
Failed <<< isaac_ros_nitros [51.2s, exited with code 2]
Aborted <<< nvblox [1min 48s]Unfortunately I have arrived at a dead end when researching solutions.
Any help would be greatly appreciated. Best regards and thank you in advance.
--
Edit: Attached CMake logfile from nvblox
Hi! Great work of your guys!
I see you guys using nvblox by drving a car around, is this car moving autonomously? If so, what is the type of this car?
Hi Team,
We are experiencing some issue while executing the vslam node :
ros2 launch nvblox_examples_bringup nvblox_vslam_realsense.launch.py
Exact error is :
[component_container-3] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[component_container-3] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
@alexmillane @hemalshahNV @jaiveersinghNV :Tagging for visibility on issue
Regards,
Rafi
Nvblox seems to be working fine for me when trying out the Isaac sim example, but is giving very bad outputs when running the realsense-based reconstruction example. I am using a Orin Dev Kit with the latest tegra version and a Realsense D455 camera.
I have also gone through the troubleshoot page, and all the required metadata and topics are getting published to Rviz. I am not sure what the issue might be, any help will be appreciated. Thanks!
Hello, I am a student who is using this library because I want to do research on navigation using isaac sim.
I found the following error during execution, and I wonder what kind of phenomenon this error was caused by.
This is the youtube link of the following error.
And error is below
2022-05-09 03:46:24 [394,582ms] [Warning] [omni.usd] Warning: in Orthonormalize at line 495 of /buildAgent/work/ca6c508eae419cf8/USD/pxr/base/gf/matrix4d.cpp -- OrthogonalizeBasis did not converge, matrix may not be orthonormal.
2022-05-09 03:46:24 [394,634ms] [Warning] [omni.usd] Warning: in Orthonormalize at line 495 of /buildAgent/work/ca6c508eae419cf8/USD/pxr/base/gf/matrix4d.cpp -- OrthogonalizeBasis did not converge, matrix may not be orthonormal.
2022-05-09 03:46:24 [394,634ms] [Warning] [omni.usd] Warning: in Orthonormalize at line 495 of /buildAgent/work/ca6c508eae419cf8/USD/pxr/base/gf/matrix4d.cpp -- OrthogonalizeBasis did not converge, matrix may not be orthonormal.
^CSimulation App Shutting Down
you wrote
Therefore, to use nvblox on jetson you have two options:
1. manually compile ROS2 Foxy and required dependent packages from source
2. or use the Isaac ROS development Docker image from [Isaac ROS Common](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common).
However, there is also a third option to do dist-upgrade on jetson that will result in 20.04 ubuntu system-wide
AV
Hello Nvidia,
During [ROS2 Setup] - [Isaac ROS Docker Setup], the below command is not working well
git clone --recurse-submodules https://gitlab-master.nvidia.com/isaac_ros/isaac_ros_nvblox.git
The response is like this
fatal: unable to access 'https://gitlab-master.nvidia.com/isaac_ros/isaac_ros_nvblox.git/': Could not resolve host: gitlab-master.nvidia.com
When I try to build ROS2 natively, when I go to the step of "git clone --recurse-submodules https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox", an error is displayed
Hi
I'm implementing nvblox and VSLAM on my robots to do a navigation application
The installation is good and I can run the examples of nvblox bag example and VSLAM realsense example
but Got Error When running the nvblox realsense and NVSLAM examples
https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_nvblox/blob/main/docs/tutorial-nvblox-vslam-realsense.md
here are error logs:
[nvblox_node-3] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[nvblox_node-3] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
[nvblox_node-3] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[nvblox_node-3] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
[nvblox_node-3] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[nvblox_node-3] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
[nvblox_node-3] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[nvblox_node-3] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
[rviz2-5] [INFO] [1664879434.204059913] [rclcpp]: signal_handler(signum=2)
[rviz2-5] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[rviz2-5] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
[rviz2-5] Warning: Invalid frame ID "odom" passed to canTransform argument target_frame - frame does not exist
[rviz2-5] at line 93 in /opt/ros/humble/src/geometry2/tf2/src/buffer_core.cpp
[component_container-1] [INFO] [1664879436.427147138] [rclcpp]: signal_handler(signum=2)
[component_container-1] [INFO] [1664879436.666174595] [rclcpp]: signal_handler(signum=2)
[component_container-1] [INFO] [1664879436.827491359] [rclcpp]: signal_handler(signum=2)
[component_container-1] [INFO] [1664879437.023495947] [rclcpp]: signal_handler(signum=2)
[component_container-1] terminate called after throwing an instance of 'rclcpp::exceptions::InvalidParameterTypeException'
[component_container-1] what(): parameter 'accel_fps' has invalid type: cannot undeclare an statically typed parameter
I can confirm the camera are running since the topic can output images.
Does anyone have ideas on how this could be solve?
BTW I'm new to docker, why does the rqt_graph doesn't work in the container although installed it?
Much appreciated if anyone could help.
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.