Comments (22)
It seems to be the problem of you dynamic library file. How do you build the tvm_runtime_op.so
file? @zouzhene
from tftvm.
Doesn’t it look like some ABI issues?
from tftvm.
@tobegit3hub
`#!/bin/bash
set -x
set -e
TF_CFLAGS=( $(python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_compile_flags()))') )
TF_LFLAGS=( $(python -c 'import tensorflow as tf; print(" ".join(tf.sysconfig.get_link_flags()))') )
g++ -g -std=c++11 -shared tvm_runtime_kernels.cc tvm_runtime_ops.cc -o tvm_runtime_op.so -fPIC ${TF_CFLAGS[@]} ${TF_LFLAGS[@]} -O2 -I${TVM_HOME}/include -I${TVM_HOME}/3rdparty/dmlc-core/include -I${TVM_HOME}/3rdparty/dlpack/include -I/usr/local/cuda/include -ltvm_runtime -L${TVM_HOME}/build -ldl -lpthread
`
Some warning like this:
In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/logging.h:25:0, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/refcount.h:22, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:21, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23, from tvm_runtime_kernels.cc:8: /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/default/logging.h:77:0: warning: "LOG" redefined #define LOG(severity) _TF_LOG_##severity ^ In file included from /usr/tvm/3rdparty/dmlc-core/include/dmlc/io.h:15:0, from /usr/tvm/include/tvm/runtime/module.h:29, from tvm_runtime_kernels.cc:5: /usr/tvm/3rdparty/dmlc-core/include/dmlc/./logging.h:257:0: note: this is the location of the previous definition #define LOG(severity) LOG_##severity.stream() ^ In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/logging.h:25:0, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/refcount.h:22, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:21, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23, from tvm_runtime_kernels.cc:8: /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/default/logging.h:89:0: warning: "VLOG" redefined #define VLOG(lvl) \ ^ In file included from /usr/tvm/3rdparty/dmlc-core/include/dmlc/io.h:15:0, from /usr/tvm/include/tvm/runtime/module.h:29, from tvm_runtime_kernels.cc:5: /usr/tvm/3rdparty/dmlc-core/include/dmlc/./logging.h:255:0: note: this is the location of the previous definition #define VLOG(x) LOG_INFO.stream() ^ In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/logging.h:25:0, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/refcount.h:22, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:21, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23, from tvm_runtime_kernels.cc:8: /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/default/logging.h:97:0: warning: "CHECK" redefined #define CHECK(condition) \ ^ In file included from /usr/tvm/3rdparty/dmlc-core/include/dmlc/io.h:15:0, from /usr/tvm/include/tvm/runtime/module.h:29, from tvm_runtime_kernels.cc:5: /usr/tvm/3rdparty/dmlc-core/include/dmlc/./logging.h:205:0: note: this is the location of the previous definition #define CHECK(x) \ ^ In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/logging.h:25:0, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/refcount.h:22, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:21, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23, from tvm_runtime_kernels.cc:8: /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/default/logging.h:251:0: warning: "CHECK_EQ" redefined #define CHECK_EQ(val1, val2) CHECK_OP(Check_EQ, ==, val1, val2) ^ In file included from /usr/tvm/3rdparty/dmlc-core/include/dmlc/io.h:15:0, from /usr/tvm/include/tvm/runtime/module.h:29, from tvm_runtime_kernels.cc:5: /usr/tvm/3rdparty/dmlc-core/include/dmlc/./logging.h:213:0: note: this is the location of the previous definition #define CHECK_EQ(x, y) CHECK_BINARY_OP(_EQ, ==, x, y) ^ In file included from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/logging.h:25:0, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/lib/core/refcount.h:22, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/platform/tensor_coding.h:21, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/resource_handle.h:19, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/allocator.h:24, from /usr/local/lib/python3.5/dist-packages/tensorflow/include/tensorflow/core/framework/op_kernel.h:23, from tvm_runtime_kernels.cc:8:
from tftvm.
@zouzhene
I guess you use pre-built tensorflow? What's your compiler version?
from tftvm.
@zhuochenKIDD I use pre-built tensorflow ,version is tensorflow-gpu==1.15.0
from tftvm.
I met the same problem, with tensorflow 2.0.0
tensorflow.python.framework.errors_impl.NotFoundError: tvm_runtime_op.so: undefined symbol: _ZN3tvm7runtime6Module12LoadFromFileERKSsS3_
from tftvm.
Seems the linking error, which so
should I link to get Module:LoadFromFile
?
from tftvm.
On these issues, could you try “nm -g tvm_runtime_op.so” to see if similar symbols exist? If so, this would be indication of compiler ABI incompatibility
from tftvm.
I removed the --shared
flag and compilation failed, saying
undefined reference to `tvm::runtime::Module::LoadFromFile(std::string const&, std::string const&)'
from tftvm.
from tftvm.
I found this. I used the pip install version of tf 2.0. Should I change it to 1?
from tftvm.
Rebuilding tvm with cmake -DCMAKE_CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0 ..
solves the problem. However there's another error for llvm's cxx11 abi.........................................
from tftvm.
@VoVAllen Perhaps you need to build llvm from source, using -D_GLIBCXX_USE_CXX11_ABI=0 : apache/tvm#521 (comment)
I guess @tobegit3hub uses self-built TensorFlow with high version gcc(>=5), so ABI incompatibility issue did not occur.
I you use pre-built TensorFlow from pip install, tf is built by gcc 4.8.5, check your env's compiler version for ABI incompatibility.
from tftvm.
It's annoying. You have to rebuilt tensorflow or llvm/tvm to solve this. Both of them took long time to do so...
from tftvm.
TVM can be viewed as two parts:
- libtvm.so: TVM compiler that does code generation, which may require LLVM for host-side code generation. The binary generated has stable pure C ABI, and guaranteed to be thread safe.
- libtvm_runtime.so: TVM runtime that can load and run operators generated by TVM compiler, which does not depend on LLVM or anything else.
Especially, TVM runtime can be run in header-only mode, which means it could be potentially build together with TensorFlow to avoid any ABI issues. In this case, I would say that it could potentially avoid having to rebuild LLVM.
from tftvm.
Sorry for the late response.
We have updated the code for the final Python API and the build script was updated to avoid conflict of re-loading tvm_runtime
. Would you try the latest code now with the instructions in README? @zhuochenKIDD @VoVAllen
from tftvm.
@tobegit3hub I have a similar issue here. In order to build correctly, I have upgrade cmake to 3.16, build with c++14 rather than c++11, and add include_directories(/usr/local/cuda/include)
. Besides, I am also using self-built tensorflow with version 1.14
And when I am running the addone example, I got this error tvm_dso_op.so: undefined symbol: _ZTIN10tensorflow8OpKernelE
.
from tftvm.
I am getting same error:
tvm_dso_op.so: undefined symbol: _ZTIN10tensorflow8OpKernelE
Have you, by any chance, found a fix already? Thanks!
from tftvm.
Yes, I rewrite the CMakelists.txt
. I use target_link_libraries
rather than target_link_options
to link the Tensorflow Library.
By the way, directly compile TVM runtime
source files into your project rather than link the prebuilt library
from tftvm.
Thanks for the replay Siyuan. If you use target_link_libraries, does that mean you are not getting link flags from tf.sysconfig.get_link_flags? It seems a bit fragile when TF changes it link flags in the future.
Meanwhile, I did some investigation and found the issue is that target_link_options place -ltensorflow_framework in front of compile targets in g++ command, this effectively discards all symbols in tensorflow_framework.so*.
I fixed the issue by building tvm_dso_op.so using Bazel, which handles library dependency ordering correctly.
from tftvm.
@gmagogsfm
Yes, you are right about the extra works need if target_link_libraries
is used. And good to know that you build your project with Bazel
from tftvm.
Thanks all for the usage. Now the patch is already merged into TVM upstream and you can build tvm with USE_TF_TVMDSOOP=ON
to use this.
The issue will be closed and only maintain in tvm codebase.
from tftvm.
Related Issues (4)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from tftvm.