williamberman / example-hip-build Goto Github PK

Example of building HIP targeting nvidia and rocm. Modified from HIP examples to build for both amd and nvidia.

This builds on amd and nvidia machines but the resulting binary hasn't been ran on an amd machine that supports rocm.

$ mkdir build-amd
$ cd build-amd
$ cmake -DHIP_PLATFORM=amd ..
$ make
$ ./bit_extract

$ mkdir build-nvidia
$ cd build-nvidia
$ cmake -DHIP_PLATFORM=nvidia ..
$ make
$ ./bit_extract
info: running on device #0 Tesla T4
info: allocate host mem (  7.63 MB)
info: allocate device mem (  7.63 MB)
info: copy Host2Device
info: launch 'bit_extract_kernel' 
info: copy Device2Host
info: check result
PASSED!

$ mkdir build-nvidia-custom-arch
$ cd build-nvidia-custom-arch
$ cmake -DHIP_PLATFORM=nvidia -DCUDA_ARCH=compute_75 ..
$ make
$ ./bit_extract
info: running on device #0 Tesla T4
info: allocate host mem (  7.63 MB)
info: allocate device mem (  7.63 MB)
info: copy Host2Device
info: launch 'bit_extract_kernel' 
info: copy Device2Host
info: check result
PASSED!

The rocm install comes with a cmake config file for hip, but it only works when targeting amd. The HIP source has cmake find modules which have code for handling targetting both nvidia and amd. The find modules could be vendored with our source but there are no docs and they give opaque errors.

The hip cmake config module defines three targets, hip::amdhip64, hip::host, and hip::device.

hip::host and hip::device are interface targets. hip::amdhip64 is an imported target for libamdhip.

You do not directly link to hip::amdhip64. hip::host links to hip::amdhip64 and hip::device links to hip::host. Link to hip::host to just use hip without compiling any GPU code. Link to hip::device to compile the GPU device code.

Docs (outdated): https://rocmdocs.amd.com/en/latest/Installation_Guide/Using-CMake-with-AMD-ROCm.html

The targets defined by the hip cmake config only target amd devices. For targeting nvidia devices, we'll make our own interface target, hip_device_nvidia, that includes the rocm and hip headers.

The platform hipcc targets is configured by the HIP_PLATFORM env var. Ideally, as we could in the shell, we would call HIP_PLATFORM=nvidia hipcc <...>. However, CMAKE_CXX_COMPILER doesn't allow configuration as such. Additionally, cmake doesn't allow setting environment variables for target builds like make does with exported variables.

We write out a file, nvidia_hipcc, into the build directory to handle configuring hipcc. nvidia_hipcc uses cmake -E env to agnostically set HIP_PLATFORM before calling hipcc.