Why is colcon not starting the build process if I omit the '--merge-install' flag after the first run? Can't we assume that the user intends to build in the same layout like last time?
So instead of forcing the user to type --merge-install every time (or use an alias or similar), couldn't colcon use the detected layout?

I found that when I move the install folder for a ROS2 workspace, and then source it, my ament_cmake packages are not being found by ros2 pkg list.

Here is the workspace I'm using to reproduce this: src.zip. The workspace builds on the ros:dashing docker image without any extra dependencies. Extract the src folder to directory workspace then from inside the workspace directory execute the following:

docker run -it -v $(pwd):/workspace ros:dashing bash -c "cd /workspace; colcon build"
docker run -it -v $(pwd):/run ros:dashing bash -c "cd /run; COLCON_CURRENT_PREFIX=/run/install source install/setup.sh; ros2 run test_cpp_package main_exe"

In comparison, this works:

docker run -it -v $(pwd):/workspace ros:dashing bash -c "cd /workspace; COLCON_CURRENT_PREFIX=/workspace/install source install/setup.sh; ros2 run test_cpp_package main_exe"

It appears that AMENT_CURRENT_PREFIX used here is not set correctly when the install folder is moved for ament_cmake packages

Based on my reading of the code I think AMENT_PREFIX_PATH needs to be set to the value of COLCON_CURRENT_PREFIX for ament cpp packages.

share/<package>/package.sh is invoked here:

https://github.com/colcon/colcon-core/blob/9d1803ecda610a411fdb98fa9b817ad0f8a64476/colcon_core/shell/template/prefix.sh.em#L117-L122

The share/<package>/package.sh sets COLCON_CURRENT_PREFIX to the correct value for the package, and what would be the correct value for AMENT_CURRENT_PREFIX:

https://github.com/colcon/colcon-core/blob/9d1803ecda610a411fdb98fa9b817ad0f8a64476/colcon_core/shell/template/package.sh.em#L63

AMENT_CURRENT_PREFIX is set for the ament package here before it is used to add the package to the `AMENT_PREFIX_PATH:

https://github.com/ament/ament_package/blob/a94116c141323671781c917c4284a583b2038ea8/ament_package/template/package_level/local_setup.sh.in#L5

I am unsure how/where the fix would be implemented. colcon_core doesn't know about ament or ament_package and ament_package doesn't know about colcon-core, so I assume the change would be in colcon-ros.

Thanks,
Matt

Getting a segfault for a build that works with catkin_make. Not sure where to go from here.

The package is https://github.com/machines-in-motion/mpi_cmake_modules

colcon --log-level 9 build

0.822s] DEBUG:colcon.colcon_core.event_handler.log_command:Invoking command in '/home/ubuntu/environment/simulation_ws/src/build/mpi_cmake_modules': /usr/bin/cmake /home/ubuntu/environment/simulation_ws/src/mpi_cmake_modules -DCATKIN_INSTALL_INTO_PREFIX_ROOT=0 -DCMAKE_INSTALL_PREFIX=/home/ubuntu/environment/simulation_ws/src/install/mpi_cmake_modules
[4.481s] INFO:colcon.colcon_core.shell:Creating environment hook '/home/ubuntu/environment/simulation_ws/src/install/mpi_cmake_modules/share/mpi_cmake_modules/hook/ros_package_path.ps1'
[4.481s] DEBUG:colcon.colcon_core.event_handler.log_command:Invoked command in '/home/ubuntu/environment/simulation_ws/src/build/mpi_cmake_modules' returned '-11': /usr/bin/cmake /home/ubuntu/environment/simulation_ws/src/mpi_cmake_modules -DCATKIN_INSTALL_INTO_PREFIX_ROOT=0 -DCMAKE_INSTALL_PREFIX=/home/ubuntu/environment/simulation_ws/src/install/mpi_cmake_modules

Trying to run it manually:

$ /usr/bin/cmake /home/ubuntu/environment/simulation_ws/src/mpi_cmake_modules -DCATKIN_INSTALL_INTO_PREFIX_ROOT=0 -DCMAKE_INSTALL_PREFIX=/home/ubuntu/environment/simulation_ws/src/install/mpi_cmake_modules
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Using CATKIN_DEVEL_PREFIX: /home/ubuntu/environment/simulation_ws/src/build/mpi_cmake_modules/devel
-- Using CMAKE_PREFIX_PATH: /opt/ros/melodic
-- This workspace overlays: /opt/ros/melodic
-- Found PythonInterp: /usr/bin/python2 (found suitable version "2.7.15", minimum required is "2")
-- Using PYTHON_EXECUTABLE: /usr/bin/python2
-- Using Debian Python package layout
-- Using empy: /usr/bin/empy
-- Using CATKIN_ENABLE_TESTING: ON
-- Call enable_testing()
-- Using CATKIN_TEST_RESULTS_DIR: /home/ubuntu/environment/simulation_ws/src/build/mpi_cmake_modules/test_results
-- Found gmock sources under '/usr/src/googletest': gmock will be built
-- Found PythonInterp: /usr/bin/python2 (found version "2.7.15")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found gtest sources under '/usr/src/googletest': gtests will be built
-- Using Python nosetests: /usr/bin/nosetests-2.7
-- catkin 0.7.17
-- BUILD_SHARED_LIBS is on
Segmentation fault (core dumped)

I have been working with a ROS 2 workspace that contains a ROS 2 Python package with a setup.py and package.xml. Before the newest release of colcon-ros, building this package and sourcing the install/setup.sh file would make the package show up as expected using ros2 pkg list.

With the newest version of colcon-core and colcon-ros, ROS 2 will not recognize the package even after building the package and sourcing the install/setup.sh file. I believe that this may be due to the ament_index folder not being generated with the newest version of colcon-ros. Is this behaviour expected, and if so is there an update to another dependency that will allow the package to be properly recognized?

There's some code in colcon_cmake which skips the package build when CMAKE_PROJECT_NAME is not set or could not be detected. When this case is triggered and the warning is printed because the cache is not readable, the code in colcon_ros tries to detect the availability of an install target, but fails to do so as this requires reading the cache as well. There appears to be an attempt to suppress this sort of error if the build invocation failed, but because of how colcon_cmake returned without an error, this logic fails and a backtrace is printed.

[2.863s] colcon.colcon_cmake.task.cmake.build WARNING Could not build CMake package 'foobar' because the CMake cache has no 'CMAKE_PROJECT_NAME' variable
Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/colcon_core/executor/__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/colcon_core/task/__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "/usr/lib/python3.9/site-packages/colcon_ros/task/ament_cmake/build.py", line 65, in build
    has_install_target = await has_target(args.build_base, 'install')
  File "/usr/lib/python3.9/site-packages/colcon_cmake/task/cmake/__init__.py", line 56, in has_target
    if 'Unix Makefiles' in generator:
TypeError: argument of type 'NoneType' is not iterable

The situation that led to this state was initially caused by user error, in which I had accidentally left the remnants of a prior invocation of CMake within the package sources. Removing these files resolved that specific issue.

I'm wondering if ROS support for new build types (let's say Cargo builds for Rust support) should be added to this repository or to a new one (colcon-ros-cargo?).

Problems with chained workspaces setup.* files.
I believe it's a regression from #53 and connected PRs

$ docker run -it --rm ubuntu:bionic
apt -qq update && apt -qq upgrade -y
echo 'Etc/UTC' > /etc/timezone &&     ln -s /usr/share/zoneinfo/Etc/UTC /etc/localtime &&     apt-get update && apt-get install -q -y tzdata && rm -rf /var/lib/apt/lists/*
apt-get update && apt-get install -q -y     bash-completion     dirmngr     gnupg2     lsb-release     python3-pip cmake libpoco-dev
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 421C365BD9FF1F717815A3895523BAEEB01FA116
echo "deb http://packages.ros.org/ros2/ubuntu `lsb_release -sc` main" > /etc/apt/sources.list.d/ros2-latest.list
apt-get update && apt-get install --no-install-recommends -y     git     python3-colcon-common-extensions
pip3 install -U     pytest-repeat     pytest-rerunfailures
mkdir -p /tmp/ros_ws1/src /tmp/ros_ws2/src /tmp/ros_ws3/src
cd /tmp/ros_ws1/src/ && git clone https://github.com/ament/ament_package
cd /tmp/ros_ws2/src/ && git clone https://github.com/ament/ament_cmake
cd /tmp/ros_ws3/src/ && git clone https://github.com/ros2/poco_vendor

docker exec -it <CONTAINER_NAME> bash
cd /tmp/ros_ws1
colcon build

docker exec -it <CONTAINER_NAME> bash
cd /tmp/ros_ws2
source /tmp/ros_ws1/install/setup.bash
colcon build

docker exec -it <CONTAINER_NAME> bash
cd /tmp/ros_ws3
source /tmp/ros_ws2/install/setup.bash
colcon build

docker exec -it <CONTAINER_NAME> bash
source /tmp/ros_ws3/install/setup.bash
colcon build

not found: "/tmp/ros_ws1/install/ament_package/local_setup.bash"

not found: "/tmp/ros_ws1/install/ament_package/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_core/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_export_definitions/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_export_include_directories/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_export_libraries/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_export_interfaces/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_export_link_flags/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_include_directories/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_libraries/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_export_dependencies/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_python/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_target_dependencies/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_test/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_auto/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_gtest/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_gmock/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_nose/local_setup.bash"
not found: "/tmp/ros_ws2/install/ament_cmake_pytest/local_setup.bash"

...
# source chained prefixes
# setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script
COLCON_CURRENT_PREFIX="/tmp/ros_ws1/install"
_colcon_prefix_chain_sh_source_script "$COLCON_CURRENT_PREFIX/local_setup.sh"

# setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script
COLCON_CURRENT_PREFIX="/tmp/ros_ws2/install"
_colcon_prefix_chain_sh_source_script "$COLCON_CURRENT_PREFIX/local_setup.sh"

# setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script
COLCON_CURRENT_PREFIX="/tmp/ros_ws1/install"
_colcon_prefix_chain_sh_source_script "$COLCON_CURRENT_PREFIX/local_setup.sh"

# setting COLCON_CURRENT_PREFIX avoids relying on the build time prefix of the sourced script
COLCON_CURRENT_PREFIX="/tmp/ros_ws2/install"
_colcon_prefix_chain_sh_source_script "$COLCON_CURRENT_PREFIX/local_setup.sh"
...

I have custom message package named "abc" that contains a message "MyMsg.msg".
Another custom message package named "use_abc_msgs" contains a message "UseMyMsg.msg" that refers to "abc/MyMsg".
I am seeing strange python errors when building this simple hierarchical message.

Starting >>> abc
[Processing: abc]
Finished <<< abc [38.7s]
Starting >>> use_abc_msgs
--- stderr: use_abc_msgs
Fatal Python error: init_sys_streams: can't initialize sys standard streams
Traceback (most recent call last):
  File "C:\Python37\lib\io.py", line 72, in <module>
**AttributeError: module 'abc' has no attribute 'ABCMeta'**
CMake Error at C:/dev/ros2share/ament_cmake_core/cmake/ament_cmake_package_templates-extras.cmake:41 (message):
  execute_process(C:/Python37/python.exe
  C:/dev/ros2share/ament_cmake_core/cmake/package_templates/templates_2_cmake.py
  C:/test/build/use_abc_msgs/ament_cmake_package_templates/templates.cmake)
  **returned error code Exit code 0xc0000409**

Call Stack (most recent call first):
  C:/dev/ros2share/ament_cmake_core/cmake/ament_cmake_coreConfig.cmake:38 (include)
  C:/dev/ros2share/ament_cmake/cmake/ament_cmake_export_dependencies-extras.cmake:15 (find_package)
  C:/dev/ros2share/ament_cmake/cmake/ament_cmakeConfig.cmake:38 (include)
  CMakeLists.txt:19 (find_package)
-

I don't understand how Python Abstract Base Classes are invoked here. I found a similar posting related to Catkin but did not see a satisfactory solution.
I have attached a zip archive,
test_custom_msg.zip, that shows this issue.

System environment:
OS: Windows 10
ROS 2: Dashing (built from sources)
CMake: 3.14.7
Python: 3.7.2

Reproduction steps:

1. Extract the zip-archive
2. Change folder to the extracted archive
3. Run local_setup.bat or setup.bash (I'm sure this is platform independent).
4. Run "colcon build --merge-install"

Any guidance will be greatly appreciated.

Thanks.

Given the growth and power of the Swift language, I’m considering building a swift client library for ROS 2. This would allow for easy, powerful and expressive code on the official platforms of Mac and Linux as well as expand support for iOS apps. The RCL repo is pretty self explanatory, but I’m trying to figure out how to integrate third party clients with the ROS 2 build system. Is there a recommended way to go about this? Unofficial clients are a big part of the ros2 plan but there’s not a lot of discussion on how to ideally implement these.

Hi, thanks for this great package.
I am having a not before seen issue, where colcon build logs the Finish and Summary of a ROS2 package, but "hangs" between 2 and 5 minutes afterwards, holding up that terminal.
Note: same hang-up occurs for failed builds...

If I CTRL-C in order to continue using that terminal, am I interrupting full completion of the process?

What could be the root cause?

ROS2 env:

   PLATFORM INFORMATION
system           : Linux
platform info    : Linux-5.4.72-microsoft-standard-WSL2-x86_64-with-glibc2.29
release          : 5.4.72-microsoft-standard-WSL2
processor        : x86_64

   RMW MIDDLEWARE
middleware name    : rmw_fastrtps_cpp

   ROS 2 INFORMATION
distribution name      : foxy
distribution type      : ros2
distribution status    : active
release platforms      : {'ubuntu': ['focal']}

I followed the install instruction on Ros2 Eloquent. If call colcon build on my workspace, I get this error

ModuleNotFoundError: No module named 'catkin_pkg'

then I tried the instruction here https://colcon.readthedocs.io/en/released/user/installation.html but it didn't help.

I tried sudo apt install python3-catkin-* but it also doesn't work.

Here is the full error log

Starting >>> raisin_soem
--- stderr: raisin_soem                         
Traceback (most recent call last):
  File "/opt/ros/eloquent/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py", line 21, in <module>
    from catkin_pkg.package import parse_package_string
ModuleNotFoundError: No module named 'catkin_pkg'
CMake Error at /opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:94 (message):
  execute_process(/home/jemin/.virtualenvs/raisim/bin/python3
  /opt/ros/eloquent/share/ament_cmake_core/cmake/core/package_xml_2_cmake.py
  /home/jemin/ros_workspace/src/raisin/raisin_soem/package.xml
  /home/jemin/ros_workspace/build/raisin_soem/ament_cmake_core/package.cmake)
  returned error code 1
Call Stack (most recent call first):
  /opt/ros/eloquent/share/ament_cmake_core/cmake/core/ament_package_xml.cmake:49 (_ament_package_xml)
  /opt/ros/eloquent/share/ament_lint_auto/cmake/ament_lint_auto_find_test_dependencies.cmake:31 (ament_package_xml)
  CMakeLists.txt:33 (ament_lint_auto_find_test_dependencies)


---
Failed   <<< raisin_soem [0.30s, exited with code 1]
                                
Summary: 0 packages finished [0.42s]
  1 package failed: raisin_soem
  1 package had stderr output: raisin_soem
  1 package not processed

Follow up to #48, when running:

# python3-colcon-cmake                       0.2.6-1    
# python3-colcon-ros                         0.3.7-1 

# workspace containing only [email protected]:locusrobotics/test_meta.git

rm -rf build install
colcon build

# Omitting this command causes tests to fail:
colcon build --cmake-target tests

colcon test
colcon test-result --all 

Am I doing something wrong?

As per REP-149, dependency tags can specify version constraints such as gte, lt, and so forth.
A previous PR added the dependency metadata (along with version constraints) to the package information: #40
However, right now there is no way to enforce this; If no APIs were broken, the build would succeed, even if the dependency's version is declared as incompatible. There could be missing/incomplete functionality which would go unnoticed because the build succeeded.

I can think of a couple of approaches to provide this kind of functionality:

1. Extend REP-149's version constraints

Support specifiers like strict_version_gte, strict_version_lt etc.
Colcon (and/or other build tools like catkin_make) would inspect the versions of the dependencies and fail the build if a strict version constraint is not met.

2. Support an optional argument to the build tools

Keep REP-149 version constraints as they are now; when invoking something like colcon build --enforce-version-constraints, the build would fail if the version requirements are not met.

I'm trying to debug something strange when migrating from catkin-tools to colcon, where the rosbash environment hook doesn't seem to get loaded properly when using colcon

EDIT: See next comment instead

Any help is appreciated!

Hello,

This issue is originally from https://forums.aws.amazon.com/thread.jspa?threadID=308176&tstart=0

This workspace builds with catkin_make_isolated but fails with colcon build.

Create workspace by doing the following:

1. Create a ros workspace and enter the src directory
2. git clone https://github.com/ros-industrial/fanuc.git
3. git clone https://github.com/ros-industrial/industrial_core.git
4. git clone https://github.com/ros-industrial-consortium/trajopt_ros
5. cd trajopt_ros
6. git checkout kinetic-devel
7. cd ../.. Should now be in root of ros directory
8. wstool init src
9. wstool merge -t src src/trajopt_ros/dependencies.rosinstall
10. wstool update -t src
11. cd src/tesseract
12. git checkout kinetic-devel
13. sudo apt-get install ros-kinetic-pr2-common
14. sudo apt install ros-kinetic-ecl
15. sudo apt-get install ros-kinetic-moveit
16. sudo apt-get install ros-kinetic-navigation
17. sudo apt-get install ros-kinetic-octomap-ros

The error is:

Starting >>> tesseract_collision
--- stderr: tesseract_collision
/home/ubuntu/environment/ros_ws/src/tesseract/tesseract_collision/src/fcl/fcl_utils.cpp:45:40: fatal error: fcl/geometry/bvh/BVH_model.h: No such file or directory
compilation terminated.
make[2]: *** [CMakeFiles/tesseract_collision_fcl.dir/src/fcl/fcl_utils.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
In file included from /home/ubuntu/environment/ros_ws/src/tesseract/tesseract_collision/include/tesseract_collision/fcl/fcl_discrete_managers.h:46:0,
                 from /home/ubuntu/environment/ros_ws/src/tesseract/tesseract_collision/src/fcl/fcl_discrete_managers.cpp:48:
/home/ubuntu/environment/ros_ws/src/tesseract/tesseract_collision/include/tesseract_collision/fcl/fcl_utils.h:48:39: fatal error: fcl/narrowphase/collision.h: No such file or directory
compilation terminated.
make[2]: *** [CMakeFiles/tesseract_collision_fcl.dir/src/fcl/fcl_discrete_managers.cpp.o] Error 1
make[1]: *** [CMakeFiles/tesseract_collision_fcl.dir/all] Error 2
make: *** [all] Error 2
---
Failed   <<< tesseract_collision        [ Exited with code 2 ]

Can you please take a look and let me know what I'm missing to get colcon build to properly link the fcl library? Or if paths have changed?

Thanks!

Hi, When compiling ros2 pkg with colcon build in zsh, some colcon parameters cannot be automatically completed. What should I do? Looking forward to your reply, thank you。

Hi @dirk-thomas,

I'm interested in adding support for reading the version requirement metadata from package.xml and adding it to the descriptor. The package manifest format supports equality operators rep-0127.

Currently, descriptor.dependencies[<stage name>] returns a list of strings which maps to package names Code Link. My first thought would be to update dependencies[<stage name>] to be a list of Dependency objects that contain name and version attributes.

This seems like a possibly breaking change to the existing build sequencing code.

• What is your policy on breaking changes like this? Could I update the relevant packages along with this one and bump their version requirements in setup.cfg?
• Do you think there is a better way include this information in the package descriptor?

Thanks for your input!
Matt

I'm building my catkin workspace (yes ROS 1) with colcon build --merge-install --symlink-install. Additionally to #147, I now noticed that the LD_LIBRARY_PATH, which should be set via <ws>/install/setup.bash, is not including <ws>/install/lib, but only /opt/ros/<distro>/lib!

$ source install/setup.bash 
$ echo $LD_LIBRARY_PATH 
/opt/ros/one/lib:/opt/ros/one/lib/x86_64-linux-gnu

Example (https://github.com/asherikov/colcon_workspace/tree/as_colcon_debug/src):

• package_a, build_type=cmake
• package_b, build_type=catkin
• package_c, build_type=catkin, depends on a and b

colcon adds package_b to CMAKE_PREFIX_PATH when building package_c, but not package_a:

DEBUG:colcon.colcon_core.event_handler.log_command:Invoking command in '/home/aleks/git/colcon_workspace/build/reldebug/package_c': CMAKE_PREFIX_PATH=/home/aleks/git/colcon_workspace/install/reldebug/package_b:${CMAKE_PREFIX_PATH} PKG_CONFIG_PATH=/home/aleks/git/colcon_workspace/install/reldebug/package_b/lib/x86_64-linux-gnu/pkgconfig:/home/aleks/git/colcon_workspace/install/reldebug/package_b/lib/pkgconfig:${PKG_CONFIG_PATH} ROS_PACKAGE_PATH=/home/aleks/git/colcon_workspace/install/reldebug/package_b/share:${ROS_PACKAGE_PATH} /usr/bin/cmake /home/aleks/git/colcon_workspace/src/package_c -DCMAKE_TOOLCHAIN_FILE=/home/aleks/git/colcon_workspace/profiles/reldebug/toolchain.cmake -DCATKIN_INSTALL_INTO_PREFIX_ROOT=0 -DCMAKE_INSTALL_PREFIX=/home/aleks/git/colcon_workspace/install/reldebug/package_c

versions:
python3-colcon-ros: Installed: 0.3.21-2
python3-colcon-core: Installed: 0.6.1-2

When sourcing a ROS1 workspace built using colcon-ros, PKG_CONFIG_PATH is not exported as expected. This then breaks using pkg_config to find ROS1 packages from an overlaid ROS2 workspace.

When creating a setup.py file, setup.cfg can be used to specify some data in a declarative manner. But the logic in RosPackageIdentification.identify ignores the setup.py file if a setup.cfg file is specified.

This came up in an ugly way for my project which defines data_files in setup.py but defines some other package data in the setup.cfg file. When colcon attempts to install the project with --symlink-install option, it completely omits the package data, since it only looks at the cfg file.

Currently, catkin_pkg conditional dependency evaluation happens pretty much as early as possible, at package-identification time:

In a "normal" colcon usage scenario, it doesn't really matter when these evaluations occur since the resulting PackageDescriptor objects are short-lived and are consumed in the same environment that creates them. However, I've been piggy-backing on colcon's discovery logic to cache these, as an alternative to rosdistro_build_cache, which means that they do become long-lived, and the consuming environment might want to access the dependency information under multiple different environment configurations (eg ROS_VERSION=1 vs ROS_VERSION=2).

I know there are a variety of ways this could be handled on my end, but I wondered if there'd be any enthusiasm for a change which would store the full condition string and result in dependency metadata, and then filter the exclusions later (possibly with a standard excluded metadata or similar)? This would allow the logic from catkin_pkg to still be used, and avoid overly-complicating colcon, but also permit me to handle this conveniently on my side.

When building in an environment where gcc is not named gcc but something different like x86_64-conda-linux-gnu-cc. colcon-ros errors with this error at the end of a build:

Traceback (most recent call last):
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/site-packages/colcon_core/executor/__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/site-packages/colcon_core/task/__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/site-packages/colcon_ros/task/catkin/build.py", line 98, in build
    additional_hooks += create_pkg_config_path_environment_hooks(
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/site-packages/colcon_ros/task/cmake/__init__.py", line 24, in create_pkg_config_path_environment_hooks
    multiarch = get_multiarch()
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/site-packages/colcon_ros/task/cmake/__init__.py", line 46, in get_multiarch
    output = subprocess.check_output(['gcc', '-print-multiarch'])
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/builds/.workspace-aa0dc32c/.env/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'gcc'

An error is caught here but not the FileNotFound error.

I don't know exactly what the best approach is. We could either use the $CC variable but that might not refer to a GCC compiler, or catch the FileNotFoundError and rely on the backup of using dpkg-architecture.

Hello Colcon contributors,

Recently I attempted to make build of ROS2 which uses colcon on my Ubuntu-16.04 x86-64 machine and yooo!, could do it ! . Now, I am curious to know whether colcon can be used for cross compiling. Let's assume we prepare the required ubuntu sysroot by apt-get install of required dependencies through qemu. Does colcon provide any mean for cross compiling like cmake has a toolchain.cmake file. If not currently supported, is it planned in near future ?

Thanks in advance

With best regards
Amey

If you have a package with both package.xml and setup.py - if the package.xml has invalid syntax, then colcon build silently falls back to a regular python build type, without even printing a warning.

I would expect this to actually fail the build, as it's a pretty serious error condition with confusing side effects. Instead it's only logged as a debug.

# COLCON_LOG_LEVEL=debug colcon build --packages-select mypkg
...
[0.216s] DEBUG:colcon.colcon_core.package_identification:Failed to parse potential ROS package manifest in'src/mypkg': Error(s) in package 'src/mypkg/package.xml':                                                                                     
The manifest contains invalid XML:                                                                                                                                                                                                                            
mismatched tag: line 4, column 12  
...
# echo $?
0

This is a feature request to have some way for ament_python packages to reference the source directory when building CPython extensions.

import os
from setuptools import setup, Extension


ext_modules = [
    Extension(
        'ape_needs_src',
        [
            'src/asdf.cpp',
        ],
        language='c++'
    ),
]


setup(
    name='destructible_capsule',
    version='0.8.3',
    author='Shane Loretz',
    author_email='[email protected]',
    ext_modules=ext_modules,
    zip_safe=False,
)

When built with colcon, the build fails complaining the file does not exist.

$ colcon build --packages-select ape_needs_src
Starting >>> ape_needs_src
[1.754s] WARNING:colcon.colcon_ros.task.ament_python.build:Package 'ape_needs_src' doesn't explicitly install the 'package.xml' file (colcon-ros currently does it implicitly but that fallback will be removed in the future)
--- stderr: ape_needs_src                   
x86_64-linux-gnu-gcc: error: src/asdf.cpp: No such file or directory
x86_64-linux-gnu-gcc: fatal error: no input files
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
---
Failed   <<< ape_needs_src	[ Exited with code 1 ]

I am able to build this package using python3 setup.py build. I tried replacing 'src/asdf.cpp' with a full path relative to the setup.py.

            os.path.abspath(os.path.join(os.path.dirname(__file__), 'src/asdf.cpp')),

However, when I build with colcon the error indicates the setup.py file is located in the build directory.

x86_64-linux-gnu-gcc: error: /home/sloretz/ws/ros2/build/ape_needs_src/src/asdf.cpp: No such file or directory

The file src/asdf.cpp needs to be found elsewhere, but I have no way of specifying that.

/home/sloretz/ws/ros2/src/ros2/rclpy/ape_needs_src/src/asdf.cpp

For now, we can only use env-variables within a expression of condition attribute for a dependency tag in Package.xml as below:

<depend condition="$ROS_VERSION == 1">roscpp</depend>

Is it possible to have a feature where we can pass in this variable as a command-line parameter instead of an environment variable?

Also, let me know if this is not the correct repo for this issue!

The colcon build command will skip directories containing a COLCON_IGNORE file, which is the expected (and desired) behavior. The colcon build command however also skips directories containing a CATKIN_IGNORE file, which is not necessarily the expected behavior.

My question is whether colcon build should really treat COLCON_IGNORE and CATKIN_IGNORE equivalently and if this is viewed as the desired behavior for colcon.

This lack of a distinction between COLCON_IGNORE and CATKIN_IGNORE seems to impose some unnecessary limitations when laying out a migration path from a ROS1 codebase to ROS2. It doesn't allow for phasing out catkin_make in a codebase having both ROS1 and ROS2 packages. Instead, it seems to assume the adoption of colcon for building all ROS package types (1 & 2) which would require updating all ROS1 CMakeLists.txt with the updated install-space from the get-go -- or otherwise forces a need to maintain migrated ROS2 components in a separate top-level folder that has its own ./src directory.

Is this lack of a distinction between COLCON_IGNORE and CATKIN_IGNORE really the intended and desired behavior for colcon build?

The function get_multiarch() calls gcc using subprocess. gcc is hard coded, and does not check the CXX env variable. This breaks a lot of build environments, including conda-smithy which uses prefixed gcc executable, and sets CXX to the correct location. This will also break cross compilers that use prefixes.

I apologize if this is not the place to report this issue but I couldn't find a better one.

As far as I can tell, some paths (PKG_CONFIG_PATH, CMAKE_MODULE_PATH, CMAKE_PREFIX_PATH) are not being exported through the environment setup scripts and hooks. That makes integration with other build tools more painful than it should be.

arjones@dfcdfc78e327:~$ find /opt/ros/galactic/ -name "*.pc"
/opt/ros/galactic/lib/pkgconfig/orocos-kdl.pc
/opt/ros/galactic/lib/pkgconfig/orocos_kdl.pc
/opt/ros/galactic/lib/pkgconfig/libzstd.pc
/opt/ros/galactic/lib/x86_64-linux-gnu/pkgconfig/console_bridge.pc
/opt/ros/galactic/lib/x86_64-linux-gnu/pkgconfig/urdfdom.pc
/opt/ros/galactic/lib/x86_64-linux-gnu/pkgconfig/urdfdom_headers.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-Overlay.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-Paging.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-MeshLodGenerator.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-PCZ.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-RTShaderSystem.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-Terrain.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-HLMS.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-Volume.pc
/opt/ros/galactic/opt/rviz_ogre_vendor/lib/pkgconfig/OGRE-Property.pc
/opt/ros/galactic/opt/yaml_cpp_vendor/lib/pkgconfig/yaml-cpp.pc
arjones@dfcdfc78e327:~$ source /opt/ros/galactic/setup.bash 
arjones@dfcdfc78e327:~$ env | egrep "PKG_CONFIG_PATH|CMAKE_MODULE_PATH|CMAKE_PREFIX_PATH"
arjones@dfcdfc78e327:~$ 

Is this a bug or am I missing something?

Steps to Reproduce:

1. Install conda Python 3.6 environment. (It is also reproducible in CPython environment.)
2. Install colcon-common-extensions.
3. Run Visual Studio developer command prompt.
4. Do colcon build against a catkin based project.

Actual Result:

(yyy) F:\workspace\noetic>colcon build
Starting >>> catkin
--- stderr: catkin
Traceback (most recent call last):
  File "C:\Users\xxx\AppData\Local\Continuum\miniconda3\envs\yyy\lib\site-packages\colcon_core\executor\__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "C:\Users\xxx\AppData\Local\Continuum\miniconda3\envs\yyy\lib\site-packages\colcon_core\task\__init__.py", line 92, in __call__
    return await task_method(*args, **kwargs)
  File "C:\Users\xxx\AppData\Local\Continuum\miniconda3\envs\yyy\lib\site-packages\colcon_ros\task\catkin\build.py", line 74, in build
    skip_hook_creation=True, additional_targets=additional_targets)
  File "C:\Users\xxx\AppData\Local\Continuum\miniconda3\envs\yyy\lib\site-packages\colcon_cmake\task\cmake\build.py", line 102, in build
    args, env, additional_targets=additional_targets)
  File "C:\Users\xxx\AppData\Local\Continuum\miniconda3\envs\yyy\lib\site-packages\colcon_cmake\task\cmake\build.py", line 225, in _build
    if not await has_target(args.build_base, target):
  File "C:\Users\xxx\AppData\Local\Continuum\miniconda3\envs\yyy\lib\site-packages\colcon_cmake\task\cmake\__init__.py", line 58, in has_target
    assert target == 'install'
AssertionError

Expected Result:
No errors

================

I am opening this issue to discuss the logic here:
https://github.com/colcon/colcon-cmake/blob/0.2.15/colcon_cmake/task/cmake/__init__.py#L57-L60
and the logic here:
https://github.com/colcon/colcon-ros/blob/0.3.13/colcon_ros/task/catkin/build.py#L68-L71

It seems that colcon-ros passed tests target by default. However, on the colcon-cmake side, it only reports the supported target install for Visual Studio generator. Any pointers how I can fix it would be helpful.

UPDATED: Add --cmake-args install to avoid the errors.

As a follow up of #79 which re-added the behavior which was removed in #74 the question remain: when should that magic behavior be removed?

Please comment with opinions when you think that should happen and also include some rational which motivates that specific proposal.

Using ROS2, colcon chains all workspaces properly, even /opt/ros:

# source chained prefixes
# setting COLCON_CURRENT_PREFIX avoids determining the prefix in the sourced script
COLCON_CURRENT_PREFIX="/opt/ros/crystal"
_colcon_prefix_chain_bash_source_script "$COLCON_CURRENT_PREFIX/local_setup.bash"

However, an eqivalent block is missing (in the first underlay) for ROS1 chains and therefore packages in /opt/ros (e.g. catkin) cannot be found during the build.

Description:

After building a ROS1 workspace, with two catkin packages, and sourcing the install env with source install/local_setup.bash (recommended according to https://media.readthedocs.org/pdf/colcon/latest/colcon.pdf, page 6), I am not able to rosrun any binary or roslaunch any launch file, getting something as RLException: [<launch_file_name>.launch] is neither a launch file in package [<package_name>] nor is [<package_name>] a launch file name The traceback for the exception was written to the log file, which means that the environment is not properly setup.

Note that I already tried as well to source setup.bash instead of local_setup.bash, with the same result.

Am I missing something here?

Thanks in advance!

Building from source on Windows 10, using the latest ROS 2 repos file (Eloquent), I see several warnings coming from colcon about missing explicit installation of package.xml files.
For example:

 colcon.colcon_ros.task.ament_python.build WARNING Package 'ament_lint' doesn't explicitly install the 'package.xml' file (colcon-ros currently does it implicitly but that fallback will be removed in the future)

I've verified that these packages do have installation commands in their setup.py (e.g.), and we don't see the warnings on Linux. I'm not sure if this is a bug in colcon-ros, or the way that these Python packages are installing their package.xml's on Windows.

What does colcon do with the cmake variable CATKIN_DEVEL_PREFIX?

These lines in moveit use it:

https://github.com/ros-planning/moveit/blob/ac55f9e252fb24321c4066d71bca4f447b55d961/moveit_ros/planning_interface/py_bindings_tools/CMakeLists.txt#L12-L16

and as a result tests that use the python bindings for the planning interface fail with an error about not being able to find the symbol _moveit_roscpp_initializer.

I know colcon doesn't have the concept of a devel space so how is this variable handled when building a catkin project?

We have an internal CI system that builds a mix of catkin and ament packages.

Following from colcon/colcon.readthedocs.org#29, the invocation for a package test is something like:

colcon build ... &&
colcon build --cmake-target tests ... || true &&   # fails for any ament packages
colcon test ...

Which is one way to make it work for both catkin and ament, however we get into messy situations if the second invocation fails for a catkin package, since it'll still try to run tests, and mask the underlying error.

Would it make sense to 'teach' colcon-ros catkin build to build the tests target as well? I can together a PR to that effect.

FYI @DLu

https://github.com/colcon/colcon-ros/releases/tag/0.3.20

This is really minor but it seems there is a release tagged 0.3.20 and it seems to contain no colcon-ros. Perhaps something went wrong?

Hi there! I'm having issues using colcon test. The details are below.

The Issue

Calling colcon test --packages-select rmf_mock_adapter_python --python-testing pytest does not seem to work.

The issue also persists if the --python-testing setuppy_test argument is used as well.

I have a toy test in the <package-dir>/tests directory, and the tests directory has an __init__.py.
But that doesn't matter because tests don't even begin to get run.

Calling colcon test without the --python-testing argument specified causes the error messages listed below to disappear. But the package fails with an [ Exited with code 2 ]

Calling pytest manually in the tests directory shows test results as expected.

Not sure if it's an issue with my set up or with colcon.

Environment Details

Ubuntu 18.04

Colcon versions

colcon-argcomplete 0.3.3: up-to-date
colcon-bash 0.4.2: up-to-date
colcon-cd 0.1.1: up-to-date
colcon-cmake 0.2.22: up-to-date
colcon-core 0.5.9: up-to-date
colcon-defaults 0.2.5: up-to-date
colcon-devtools 0.2.2: up-to-date
colcon-library-path 0.2.1: up-to-date
colcon-metadata 0.2.4: up-to-date
colcon-notification 0.2.13: up-to-date
colcon-output 0.2.9: up-to-date
colcon-package-information 0.3.3: up-to-date
colcon-package-selection 0.2.6: up-to-date
colcon-parallel-executor 0.2.4: up-to-date
colcon-pkg-config 0.1.0: up-to-date
colcon-powershell 0.3.6: up-to-date
colcon-python-setup-py 0.2.5: up-to-date
colcon-recursive-crawl 0.2.1: up-to-date
colcon-ros 0.3.17: up-to-date
colcon-test-result 0.3.8: up-to-date
colcon-zsh 0.4.0: up-to-date

setup.py

Relevant sections. Everything else builds normally, only the test messes up.

setup(
    name=package_name,
    version=__version__,
    packages=[],
    author='methylDragon',
    author_email='[email protected]',
    url='',
    description='Python bindings for rmf_mock_adapter',
    long_description='',
    ext_modules=[CMakeExtension(package_name)],
    setup_requires=['pybind11>=2.5.0', 'pytest-runner'],
    cmdclass={'build_ext': CMakeBuild},
    zip_safe=False,
    entry_points={'console_scripts': []},
    license='Apache License, Version 2.0',
    install_requires=['setuptools'],
    data_files=[
        ('share/ament_index/resource_index/packages',
            ['resource/' + package_name]),
        ('share/' + package_name, ['package.xml'])
    ],
    tests_require=['pytest'],
    test_suite="tests"
)

Error messages

Starting >>> rmf_mock_adapter_python
--- stderr: rmf_mock_adapter_python                   
Traceback (most recent call last):
  File "/home/methyldragon/.local/lib/python3.6/site-packages/colcon_core/task/python/test/__init__.py", line 75, in test
    return await extension.step(self.context, env, setup_py_data)
AttributeError: 'NoneType' object has no attribute 'step'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/methyldragon/.local/lib/python3.6/site-packages/colcon_core/executor/__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "/home/methyldragon/.local/lib/python3.6/site-packages/colcon_core/task/__init__.py", line 92, in __call__
    return await task_method(*args, **kwargs)
  File "/home/methyldragon/.local/lib/python3.6/site-packages/colcon_ros/task/ament_python/test.py", line 29, in test
    return await extension.test()
  File "/home/methyldragon/.local/lib/python3.6/site-packages/colcon_core/task/python/test/__init__.py", line 81, in test
    "'{extension.STEP_TYPE}': {e}\n{exc}".format_map(locals()))
AttributeError: 'NoneType' object has no attribute 'STEP_TYPE'
---
Failed   <<< rmf_mock_adapter_python	[ Exited with code 1 ]

Summary: 0 packages finished [0.50s]
  1 package failed: rmf_mock_adapter_python
  1 package had stderr output: rmf_mock_adapter_python
'NoneType' object has no attribute 'STEP_TYPE'

#55 introduced a warning for catkin install spaces that were created without local_setup.* (before ros/catkin#993).

The warning gets issued if .catkin is present, but local_setup.* is not.
However, the (default) isolated catkin layout includes .catkin for each isolated package and therefore triggers the warning in overlay workspaces.

Underlay:

. /opt/ros/melodic/setup.sh
mkdir -p /tmp/ws/src
cd /tmp/ws/src
catkin_create_pkg pkg_a
catkin_create_pkg pkg_b
cd /tmp/ws/
colcon build

Resulting layout:

tree -a -L 2 /tmp/ws/install
/tmp/ws/install
├── COLCON_IGNORE
├── .colcon_install_layout
├── local_setup.bash
├── local_setup.ps1
├── local_setup.sh
├── _local_setup_util.py
├── local_setup.zsh
├── pkg_a
│   ├── .catkin
│   ├── lib
│   └── share
├── pkg_b
│   ├── .catkin
│   ├── lib
│   └── share
├── setup.bash
├── setup.ps1
├── setup.sh
└── setup.zsh

6 directories, 13 files

Overlay:

. /tmp/ws/install/setup.sh
mkdir -p /tmp/ws2/src
cd /tmp/ws2/src
catkin_create_pkg pkg_c
cd /tmp/ws2/
colcon build

Output

Starting >>> pkg_c   
[0.226s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/tmp/ws/install/pkg_b' in the environment variable CMAKE_PREFIX_PATH seems to be a catkin workspace but it doesn't contain any 'local_setup.*' files. Maybe the catkin version is not up-to-date?
[0.226s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/tmp/ws/install/pkg_a' in the environment variable CMAKE_PREFIX_PATH seems to be a catkin workspace but it doesn't contain any 'local_setup.*' files. Maybe the catkin version is not up-to-date?
Finished <<< pkg_c [2.23s]                               

Summary: 1 package finished [2.36s]

Not sure if .catkin should be added there, though.
If this is intendeded, the warning should only get printed if .catkin and setup.* are present, but local_setup.* is not.

I haven't got a MWE for this, but we're seeing an issue sometimes where the CMake invocation fails, but the error message is unhelpful because it pertains to make help having failed. I believe this logic here should check the value of rc before calling has_target, or else the inner get_makefile_targets function should catch the exception and return an empty list in that case:

Example:

-- Configuring incomplete, errors occurred!
See also "[...]/ws/build/python_orocos_kdl/CMakeFiles/CMakeOutput.log".
See also "[...]/ws/build/python_orocos_kdl/CMakeFiles/CMakeError.log".
Traceback (most recent call last):
  File "[...]/colcon-env/lib/python3.8/site-packages/colcon_core/executor/__init__.py", line 91, in __call__
    rc = await self.task(*args, **kwargs)
  File "[...]/colcon-env/lib/python3.8/site-packages/colcon_core/task/__init__.py", line 93, in __call__
    return await task_method(*args, **kwargs)
  File "[...]/colcon-env/lib/python3.8/site-packages/colcon_ros/task/catkin/build.py", line 82, in build
    if await has_target(args.build_base, 'install'):
  File "[...]/colcon-env/lib/python3.8/site-packages/colcon_cmake/task/cmake/__init__.py", line 57, in has_target
    return target in await get_makefile_targets(path)
  File "[...]/colcon-env/lib/python3.8/site-packages/colcon_cmake/task/cmake/__init__.py", line 77, in get_makefile_targets
    output = await check_output([
  File "[...]/colcon-env/lib/python3.8/site-packages/colcon_core/subprocess.py", line 109, in check_output
    assert not rc, \
AssertionError: Expected ['/usr/bin/cmake', '--build', '[...]/ws/build/python_orocos_kdl', '--target', 'help'] to pass: make: *** No rule to make target 'help'.  Stop.

FYI @iwanders

colcon-ros maintains ROS_PACKAGE_PATH for ROS1/catkin packages, but not for ROS2 packages.

Without this, ROS2 users of rosdep must maintain ROS_PACKAGE_PATH by hand.

See: https://answers.ros.org/question/315296/crystal-rosdep-cannot-find-all-required-resources/

Error when trying to have one catkin workspace (via catkin_tools) extend another catkin workspace built by colcon, due to a missing env.sh file.

error: could not configure catkin workspace: Cannot load environment from resultspace "/opt/locus/colcon-test/bot" because the required setup file "/opt/locus/colcon-test/bot/env.sh" does not exist.

It appears that colcon does not generate an env.sh file in the install space. Is this intentional?

Referencing some of the behavior described in #16

The behavior of automatically setting the CATKIN_INSTALL_INTO_PREFIX_ROOT variable in a ROS2-only workspace leaves you with a bunch of unused cmake variable warnings:

--- stderr: package_name                                                                                   
CMake Warning:
  Manually-specified variables were not used by the project:

    CATKIN_INSTALL_INTO_PREFIX_ROOT

I silence these in each individual project with:

unset( CATKIN_INSTALL_INTO_PREFIX_ROOT )

but was wondering if there was a way to avoid having to manually specify this in each ROS2 CMakeLists.txt to avoid the additional noise in our build logs? If not, maybe supporting compatibility with catkin_tools in a mixed workspace could be placed behind some flag/option?

Slightly related to #110, in my opinion this is wrong behaviour:

None of catkin_make, catkin_make_isolated, or catkin_tools has defaulted to building test binaries. It's very surprising as a user to run colcon build and have it be building targets other than what is expressed in the CMakeLists.txt as being part of ALL.

I'm very much in support of having "run tests" be a first-class action (it never was in catkin_tools, regrettably), but I feel that building the tests should be something done just ahead of running them, not as something which must always be explicitly disabled using a flag.

I'm building ROS1 packages, e.g. moveit_calibration with colcon build --symlink-install. There we use versioned libraries:
set_target_properties(${MOVEIT_LIB_NAME} PROPERTIES VERSION "${${PROJECT_NAME}_VERSION}")

However, this causes colcon to symlink libs into the wrong folder (package instead of package/lib) and w/o symlinking .so:

> find . -iname "moveit_handeye_calibration_rviz_plugin.so*"
./install/moveit_handeye_calibration_rviz_plugin/moveit_handeye_calibration_rviz_plugin.so.0.1.0
./build/moveit_handeye_calibration_rviz_plugin/devel/lib/moveit_handeye_calibration_rviz_plugin.so.0.1.0
./build/moveit_handeye_calibration_rviz_plugin/devel/lib/moveit_handeye_calibration_rviz_plugin.so

When dropping the VERSION setting, but still performing a symlink install, it works:

> find . -iname "moveit_handeye_calibration_rviz_plugin.so*"
./install/moveit_handeye_calibration_rviz_plugin/lib/libmoveit_handeye_calibration_rviz_plugin.so
./build/moveit_handeye_calibration_rviz_plugin/devel/lib/libmoveit_handeye_calibration_rviz_plugin.so

A --merge-install with versioned libs works as expected as well:

> find . -iname "libmoveit_handeye_calibration_rviz_plugin*" | xargs ls -1la
lrwxrwxrwx 1 rhaschke nistaff      29 Nov  7 16:00 ./build/moveit_handeye_calibration_rviz_plugin/devel/lib/libmoveit_handeye_calibration_rviz_plugin.so -> libmoveit_handeye_calibration_rviz_plugin.so.0.1.0
-rwxr-x--x 1 rhaschke nistaff 2763456 Nov  7 16:00 ./build/moveit_handeye_calibration_rviz_plugin/devel/lib/libmoveit_handeye_calibration_rviz_plugin.so.0.1.0
lrwxrwxrwx 1 rhaschke nistaff      29 Nov  7 16:00 ./install/lib/libmoveit_handeye_calibration_rviz_plugin.so -> libmoveit_handeye_calibration_rviz_plugin.so.0.1.0
-rw-r--r-- 1 rhaschke nistaff 2763456 Nov  7 16:00 ./install/lib/libmoveit_handeye_calibration_rviz_plugin.so.0.1.0

So, it seems the only missing use case is versioned libs and a symlinked install.

Since the prefix_chain.sh has been renamed to setup.sh there is an infinite recursion when building a ROS 1 workspace with --merge-install:

colcon build -h

states that

  --catkin-skip-building-tests
                        By default the 'tests' target of 'catkin' packages is
                        invoked. If running 'colcon test' later isn't intended
                        this can be skipped

However I noticed this when building https://github.com/agutenkunst/minimal_test_package

colcon build --cmake-target tests && colcon test && colcon test-result --all --verbose

works but

colcon build && colcon test && colcon test-result --all --verbose

fails with

Starting >>> src     
[0.239s] WARNING:colcon.colcon_ros.prefix_path.catkin:The path '/home/alex/catkin_ws_minimal_coverage/install/src' in the environment variable CMAKE_PREFIX_PATH doesn't exist
Finished <<< src [3.81s]                       

Summary: 1 package finished [3.93s]
Starting >>> src     
--- stderr: src                   
Errors while running CTest
---
Finished <<< src [0.39s]	[ with test failures ]

Summary: 1 package finished [0.56s]
  1 package had stderr output: src
  1 package had test failures: src
build/src/Testing/20200707-1519/Test.xml: 2 tests, 0 errors, 1 failure, 0 skipped
- _ctest_minimal_test_package_gtest_moduleB_gtest
  <<< failure message
    /bin/sh: 1: /home/alex/catkin_ws_minimal_coverage/build/src/devel/lib/minimal_test_package/moduleB_gtest: not found
    Cannot find results, writing failure results to '/home/alex/catkin_ws_minimal_coverage/build/src/test_results/minimal_test_package/MISSING-gtest-moduleB_gtest.xml'
    -- run_tests.py: execute commands
      /home/alex/catkin_ws_minimal_coverage/build/src/devel/lib/minimal_test_package/moduleB_gtest --gtest_output=xml:/home/alex/catkin_ws_minimal_coverage/build/src/test_results/minimal_test_package/gtest-moduleB_gtest.xml
    -- run_tests.py: verify result "/home/alex/catkin_ws_minimal_coverage/build/src/test_results/minimal_test_package/gtest-moduleB_gtest.xml"
  >>>
build/src/test_results/minimal_test_package/MISSING-gtest-moduleB_gtest.xml: 1 test, 0 errors, 1 failure, 0 skipped
- Results test_ran
  <<< failure message
    Unable to find test results for gtest-moduleB_gtest.xml, test did not run. Expected results in /home/alex/catkin_ws_minimal_coverage/build/src/test_results/minimal_test_package/gtest-moduleB_gtest.xml
  >>>
build/src/test_results/minimal_test_package/nosetests-test.moduleA_unittest.py.xml: 1 test, 0 errors, 0 failures, 0 skipped

Summary: 4 tests, 0 errors, 2 failures, 0 skipped

My versions are

colcon-argcomplete 0.3.3: up-to-date
colcon-bash 0.4.2: up-to-date
colcon-cd 0.1.1: up-to-date
colcon-cmake 0.2.23: up-to-date
colcon-core 0.5.10: up-to-date
colcon-defaults 0.2.5: up-to-date
colcon-devtools 0.2.2: up-to-date
colcon-library-path 0.2.1: up-to-date
colcon-metadata 0.2.4: up-to-date
colcon-notification 0.2.13: up-to-date
colcon-output 0.2.11: up-to-date
colcon-package-information 0.3.3: up-to-date
colcon-package-selection 0.2.7: up-to-date
colcon-parallel-executor 0.2.4: up-to-date
colcon-pkg-config 0.1.0: up-to-date
colcon-powershell 0.3.6: up-to-date
colcon-python-setup-py 0.2.5: up-to-date
colcon-recursive-crawl 0.2.1: up-to-date
colcon-ros 0.3.18: up-to-date               <------------------------- !
colcon-test-result 0.3.8: up-to-date
colcon-zsh 0.4.0: up-to-date

I keep getting the below error when running rviz2

terminate called after throwing an instance of 'ament_index_cpp::PackageNotFoundError'
  what():  package 'rviz_common' not found, searching: [/home/user/ws/install/pkg1, /home/user/ws/install/pkg2, /home/user/ws/install/]

It was working before but after removing a workspace it kept giving me a warning

WARNING:colcon.colcon_core.prefix_path.colcon:The path '/home/user/old_ws/install' in the environment variable COLCON_PREFIX_PATH doesn't exist
WARNING:colcon.colcon_core.prefix_path.colcon:The path '/home/user/old_ws/install' in the environment variable AMENT_PREFIX_PATH doesn't exist
WARNING:colcon.colcon_core.prefix_path.colcon:The path '/home/user/old_ws/install' in the environment variable CMAKE_PREFIX_PATH doesn't exist

I edited these paths manually to try removing the errors and I think it screwed things up. Is there a way to completely reset everything/clean everything and start from scratch?

I'm running some tests with colcon test --pytest-with-coverage on rolling.

This automatically enables the options --cov-report html:<some path> and --cov-report xml:<some path>.

However, there is also a --cov-report term option that should display test results to console, which I want to add manually using the --pytest-args option.

If I do the following, the results are still not displayed:

colcon test --event-handlers console_direct+ --pytest-with-coverage --pytest-args " --cov-report term"

The equivalent command that is run, per the console logs, is:

1: -- run_test.py: invoking following command in '/home/sebastian/pyrobosim_ws/src/pyrobosim/pyrobosim_ros':
1:  - /home/sebastian/python-virtualenvs/pyrobosim/bin/python3.10 -u -m pytest
 /home/sebastian/pyrobosim_ws/src/pyrobosim/pyrobosim_ros/test/test_ros_conversions.py 
-o cache_dir=/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test/ament_cmake_pytest/test_ros_conversions/.cache 
--junit-xml=/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test_results/pyrobosim_ros/test_ros_conversions.xunit.xml 
--junit-prefix=pyrobosim_ros --cov=/home/sebastian/pyrobosim_ws/src/pyrobosim/pyrobosim_ros/test 
--cov-report=html:/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test/pytest_cov/test_ros_conversions/coverage.html 
--cov-report=xml:/home/sebastian/pyrobosim_ws/build/pyrobosim_ros/test/pytest_cov/test_ros_conversions/coverage.xml 
--cov-branch

As you see, the --cov-report term text is nowhere to be found in the above snippet.

I then realized if I add any arguments, like --pytest-args " --hello", not even that gets reflected.

Am I missing something, or does the --pytest-args option not currently work?