libLLVMHipSpvPasses.so: undefined symbol llvm::convertConstantExprsToInstructions about chipstar HOT 20 CLOSED

Looking at the source code of Clang driver, it seems to invoke opt with absolute path, and the pass plugin is also an absolute path.

Possibly something in LD_LIBRARY_PATH makes Clang load the wrong library, in which case it's not our problem, but we could at least document it.

from chipstar.

I had two versions of clang14 - one with openmmp and one without. The one without openmp worked fine whereas the openmp version gave me this error. Quite strange really since there shouldn't be any other differences between my installations

from chipstar.

@pvelesko do you still have the system and the clang14 installation where this happens ? can you try running the opt command with LD_DEBUG=all ? LD_DEBUG=all .../path/to/opt [args...] 2>/tmp/opt.log and then attach the log

from chipstar.

I ran the following command which did not return any errors but the overall build still failed.
LD_DEBUG=all /gpfs/jlse-fs0/users/pvelesko/install/clang/clang14/clang14-spirv-omp/bin/opt ./bitcode/BC/nearbyintD.bc -O3 -o ./test.bc 2>~/opt.log
opt.log

@franz

from chipstar.

I'm also running into this problem.

from chipstar.

@pvelesko the opt command needs the plugin options: -load-pass-plugin /path/to/chip_build_dir/lib/libLLVMHipSpvPasses.so -passes=hip-post-link-passes, otherwise it just does normal LLVM optimizations.

from chipstar.

@jbrodman hi, do you also use OpenMP-enabled build of LLVM ? do you have multiple LLVM installations ?

from chipstar.

LD_DEBUG=all /gpfs/jlse-fs0/users/pvelesko/install/clang/clang14/clang14-spirv-omp/bin/opt ./bitcode/BC/nearbyintD.bc -O3 -o -load-pass-plugin /path/to/chip_build_dir/lib/libLLVMHipSpvPasses.so -passes=hip-post-link-passes ./test.bc 2>~/opt.log 

opt2.log

@franz

from chipstar.

I just rebuilt w/o OpenMP - no help.

So I'm mucking around with this - it doesn't seem like libLLVMHipSpvPasses.so is actually linked against LLVM anywhere:
[15/270] : && /home/jbrodman/chip/install/bin/clang++ -fPIC -Wno-duplicate-decl-specifier -Wno-tautological-constant-compare -Wno-c++20-extensions -Wno-un
used-result -Wno-delete-abstract-non-virtual-dtor -Wno-deprecated-declarations -Wunused-command-line-argument -O3 -DNDEBUG -shared -o lib/libLLVMHipSpvP
asses.so llvm_passes/CMakeFiles/LLVMHipPasses.dir/HipPasses.cpp.o llvm_passes/CMakeFiles/LLVMHipPasses.dir/HipDynMem.cpp.o llvm_passes/CMakeFiles/LLVMHipPa
sses.dir/HipStripUsedIntrinsics.cpp.o llvm_passes/CMakeFiles/LLVMHipPasses.dir/HipDefrost.cpp.o llvm_passes/CMakeFiles/LLVMHipPasses.dir/HipPrintf.cpp.o ll
vm_passes/CMakeFiles/LLVMHipPasses.dir/HipGlobalVariables.cpp.o llvm_passes/CMakeFiles/LLVMHipPasses.dir/HipTextureLowering.cpp.o llvm_passes/CMakeFiles/LL
VMHipPasses.dir/HipAbort.cpp.o

Is this right? It seems like it ought to get linked against LLVM somehow. The missing symbol can be found in libLLVMCore.a in my build of LLVM 15.

$ nm libLLVMCore.a | rg convertConstant
12090:0000000000000000 T _ZN4llvm34convertConstantExprsToInstructionsEPNS_11InstructionEPNS_12ConstantExprEPNS_15SmallPtrSetImplIS1_EE

from chipstar.

Update: Yeah - think something is bad in the build configuration for chip-spv - it seems to be not linking against static libs that it should be.

I just rebuilt LLVM - but this time with -DLLVM_BUILD_LLVM_DYLIB=ON -DLLVM_LINK_LLVM_DYLIB=ON - previously I was using the default static lib build of LLVM - and.....

[270/270] Linking CXX executable samples/cuda_samples/cuda-reduction

Success!

Trying to build all the tests fails horribly, but that's a problem for another day...
[1/1035] Building CXX object catch/unit/deviceLib/SinglePrecisionIntrinsics/CMakeFiles/__cosf.dir/__cosf.cc.o
FAILED: catch/unit/deviceLib/SinglePrecisionIntrinsics/CMakeFiles/__cosf.dir/__cosf.cc.o
/home/jbrodman/chip/install/bin/clang++ -I/home/jbrodman/chip/chip-spv/HIP/tests/catch/./include -I/home/jbrodman/chip/hipinstall/include -I/home/jbrodman/chip/chip-spv/HIP/tests/catch/external/picojson -I/home/jbrodman/chip/chip-spv/HIP/include -I/home/jbrodman/chip/chip-spv/include -I/home/jbrodman/chip/chip-spv/HIP/tests/catch/external/Catch2/include -I/home/jbrodman/chip/chip-spv/include/cuspv -I/home/jbrodman/chip/chip-spv/HIP/tests/catch/external/Catch2/single_include -Wno-duplicate-decl-specifier -Wno-tautological-constant-compare -Wno-c++20-extensions -Wno-unused-result -Wno-delete-abstract-non-virtual-dtor -Wno-deprecated-declarations -Wunused-command-line-argument -O3 -DNDEBUG -fPIE -Wno-format-extra-args -std=c++17 -MD -MT catch/unit/deviceLib/SinglePrecisionIntrinsics/CMakeFiles/__cosf.dir/__cosf.cc.o -MF catch/unit/deviceLib/SinglePrecisionIntrinsics/CMakeFiles/__cosf.dir/__cosf.cc.o.d -o catch/unit/deviceLib/SinglePrecisionIntrinsics/CMakeFiles/__cosf.dir/__cosf.cc.o -c /home/jbrodman/chip/chip-spv/HIP/tests/catch/unit/deviceLib/SinglePrecisionIntrinsics/__cosf.cc
In file included from /home/jbrodman/chip/chip-spv/HIP/tests/catch/unit/deviceLib/SinglePrecisionIntrinsics/__cosf.cc:1:
In file included from /home/jbrodman/chip/chip-spv/HIP/tests/catch/unit/deviceLib/SinglePrecisionIntrinsics/../device_tests_common.hh:1:
In file included from /home/jbrodman/chip/chip-spv/HIP/tests/catch/./include/hip_test_common.hh:24:
In file included from /home/jbrodman/chip/chip-spv/HIP/tests/catch/./include/hip_test_context.hh:24:
/home/jbrodman/chip/hipinstall/include/hip/hip_runtime.h:77:2: error: ("Must define exactly one of HIP_PLATFORM_AMD, HIP_PLATFORM_NVIDIA or HIP_PLATFORM_SPIRV");
#error("Must define exactly one of HIP_PLATFORM_AMD, HIP_PLATFORM_NVIDIA or HIP_PLATFORM_SPIRV");
^

from chipstar.

("Must define exactly one of HIP_PLATFORM_AMD, HIP_PLATFORM_NVIDIA or HIP_PLATFORM_SPIRV");
#error("Must define exactly one of HIP_PLATFORM_AMD, HIP_PLATFORM_NVIDIA or HIP_PLATFORM_SPIRV");

Would imply that something went terribly wrong with CMake configuration. Can you share your CMake configuration log? @jbrodman

from chipstar.

which file is that?

from chipstar.

@jbrodman CHIP-SPV/build/CMakeCache.txt + whatever output you get from running cmake configure

from chipstar.

it doesn't seem like libLLVMHipSpvPasses.so is actually linked against LLVM anywhere

That is actually how the LLVM manual on pass plugin development does it (without linking to LLVM):

add_library(LLVMPassname MODULE Pass.cpp)

It seems like it ought to get linked against LLVM somehow.

I have tried to linking the plugin to libLLVMCore.a, and the problem is, that it also links in static global variables from libLLVMCore. Then when opt tries to load the plugin, it crashes because they conflict with the static global variables existing in the opt binary:

opt: CommandLine Error: Option 'enable-fs-discriminator' registered more than once!
LLVM ERROR: inconsistency in registered CommandLine options
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.	Program arguments: /home/michal/0/LLVM_150_STATIC/bin/opt /tmp/hipmath-generic-link-19658d.bc -load-pass-plugin /tmp/build/lib/libLLVMHipSpvPasses.so -passes=hip-post-link-passes -o /tmp/hipmath-generic-lower-eae143.bc

After some more experiments, i have come up with an hacky workaround. I haven't found what's the official way of linking plugins to static LLVM builds. It's also possible that the problem is only related to this particular symbol (IOW a bug in LLVM).

from chipstar.

dump.txt is the output of conf.

dump.txt
CMakeCache.txt

from chipstar.

@jbrodman very strange that things are not compiling for you. CMake looks good. This line -- Generated HIP_OFFLOAD_COMPILE_OPTIONS: -D__HIP_PLATFORM_SPIRV__= -x hip --target=x86_64-linux-gnu -Xclang -no-opaque-pointers --offload=spirv64 -nohipwrapperinc --hip-path=/home/jbrodman/chip/hipinstall defines the platform via this flag -D__HIP_PLATFORM_SPIRV__

Can you try building again this time via make VERBOSE=1 and paste the result please?

from chipstar.

To be clear - I can build chip-spv itself now after finding out the LLVM shared lib issue. The build the tests step fails:

make build_tests_standalone (ninja build_tests_standalone)
tests.txt

from chipstar.

@jbrodman that doesn't look right.. it's missing the GPU offload string which was successfully generated in the previous step you sent me.
Which distro are you using?
Can you try a clean recursive clone before we dive deeper into investigating why it's failing?

from chipstar.

Ubuntu 22.04.

Sure.

from chipstar.

closing since a workaround was implemented for the original issue. @jbrodman please open another ticket if you are still having issues building the tests.

from chipstar.