xiangpenghao / cxx-cmake-example Goto Github PK

An example repo to setup cxx (Rust FFI library) with the CMake build system.

CMake 27.78% C++ 28.08% Rust 34.32% Dockerfile 9.82%

rust cpp cxx cmake ffi

cxx-cmake-example's Introduction

CXX with CMake build system

This is an example repo to setup cxx with the cmake build system.

The official demo used cargo to orchestrate the two build systems and place the main function inside the rust project.

In a lot of other applications, however, we want to embed rust into a large cpp project where we don't have a chance to choose build systems. This template repo shows how to use cmake with cxx.

The cmake files do the following things:

Call cargo build [--release] to build the static library and the necessary c++ header/source files
Create a library from the cxx generated source and link to the rust static library
Link and include the libraray to the corresponding targets

Demo

docker build . --no-cache --progress plain

Cross-language LTO (linking time optimization)

Why?

Calling rust function from c++ (and vice versa) is not zero-overhead because the LLVM optimizer by default will not optimize across shared libraries, let alone across different languages.

LTO (linking time optimization) allows the optimizers to perform optimization during linking time (instead of compile time), thus enables cross library optimization.

How?

cmake -DENABLE_LTO=ON -DCMAKE_BUILD_TYPE=Release ..
make -j
./main

The -DENABLE_LTO=ON will compile and link both libraries with proper parameters. Note that cross language LTO between rust and c/c++ is only possible with clang toolchain, meaning that you need to have a very recent clang/lld installed.

Example output

With LTO

Points {
    x: [
        1,
        2,
        3,
    ],
    y: [
        4,
        5,
        6,
    ],
}
"cpp expert!"
Calling rust function, time elapsed: 100 ns.
Calling c++ function, time elapsed: 100 ns.

Without LTO

Points {
    x: [
        1,
        2,
        3,
    ],
    y: [
        4,
        5,
        6,
    ],
}
"cpp expert!"
Calling rust function, time elapsed: 1176600 ns.
Calling c++ function, time elapsed: 100 ns.

Credits

The cmake files are largely inspired by Using unsafe for Fun and Profit.

I learned a lot about cross language LTO from this post: Closing the gap: cross-language LTO between Rust and C/C++

cxx-cmake-example's People

Contributors

Stargazers

Watchers

Forkers

dtolnay-contrib rikharthu ruvi-d jacg nub adi-g15 watch-later grayish sears anoulis arsenal591 pjiwm loongs-zhang aweffr silvanshade ssh352 harry-anderson

cxx-cmake-example's Issues

Missing target to generate librust_part.a

Hey, I was trying to use this project as a template to generate a static lib from a crate and a cpp executable and found that the executable target could not be generated due to the missing librust_part.a. If the target for the rust lib was build first everything went fine in a second stage when the executable was built.

I added ${RUST_PART_LIB} as an OUTPUT to the add_custom_command directive in the rust_part/CMakeLists.txt which lets cmake know that librust_part.a is generated by that command. Now it appears to build everything alright in one go.

Is there any reason why this could be not OK?

sccache integration

sccache is a compile caching program that supports both rustc and clang. It would be great for this minimal example to demonstrate how to use sccache for both languages on CI.

--emit=llvm-ir may be required for LTO

I only got LTO to inline the rust functions by adding --emit=llvm-ir to the RUSTFLAGS.

Not sure if this is a bug in llvm/rust, but I had a rather long journey of: trying in various containers, compiling rust myself with patches and external llvm, installing a archlinux-vm to try their custom-built rustc without success of getting the cross language LTO to work. I now wanted to debug this by looking at the IR, but just activating it with --emit=llvm-ir finally made it inline the function. I have no idea why the -Clinker-plugin-lto flag is not enough here, but this info might help other people trying to get this to work.

My setup is currently on Ubuntu 21.04 with rust stable from rustup and llvm-12 from the llvm repo:

% rustc --version --verbose
rustc 1.53.0 (53cb7b09b 2021-06-17)
binary: rustc
commit-hash: 53cb7b09b00cbea8754ffb78e7e3cb521cb8af4b
commit-date: 2021-06-17
host: x86_64-unknown-linux-gnu
release: 1.53.0
LLVM version: 12.0.1

% clang --version
Ubuntu clang version 12.0.1-++20210630032550+fed41342a82f-1~exp1~20210630133300.126
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

Build failed: `signal: 11, SIGSEGV: invalid memory reference`

I tried to build the project with this setting:

But it failed with a long log, I think this is the relevant part:

error: failed to run custom build command for `scratch v1.0.1`

Caused by:
  process didn't exit successfully: `/home/hamid/Desktop/cmake_tutor/cxx-cmake-example/build/rust_part/release/build/scratch-6d8128b52c896f83/build-script-build` (signal: 11, SIGSEGV: invalid memory reference)

Here is my setup information:

OS: Pop OS (ubuntu) 22.04
cmake: 3.22.1
rustc: 1.60.0-beta.4
clang-13: 13.0.1-2ubuntu2
lld-13: Ubuntu LLD 13.0.1 (compatible with GNU linkers)
clang: 14.0.0-1ubuntu1
lld: Ubuntu LLD 14.0.0 (compatible with GNU linkers)

Sorry if it is a trivial problem, I'm not familiar with c++ tools.

Windows and macOS CI

It would be really helpful to have an example for how to get this working on CI on Windows and macOS. It is not obvious how to do this considering the requirement for rustc and clang to be using the same version of LLVM. Ideally most of the setup steps should be in standalone shell scripts (maybe PowerShell on Windows?) that could be easily run on any CI system or to setup a local development environment. Taking it a step further, a GitHub Actions action could be made to set up clang and rustc for cross-language LTO and use caching on GitHub Actions for the binaries of the compilers.