tqchen / tinyflow Goto Github PK

This is really great. Any plan to add a lua binding for tinyflow?

!

import tinyflow as tf

v0 = tf.Variable(tf.zeros([10, 10]))
v1 = tf.Variable(tf.ones_like(v0))
sess = tf.Session()
sess.run(tf.initialize_all_variables())

The code above will occur an uninitialized variable error. It is because that v0 will be added into read_var_nids due to ones_like, which is uninitialized at this moment and will be checked. (https://github.com/tqchen/tinyflow/blob/master/src/session.cc#L328)

op in nn_module group will own the same attr which has been set.

and the line https://github.com/tqchen/tinyflow/blob/master/src/op_nn.cc#L97 set TBackwardNeedOutputs to be true so that all ops with .include("nn_module") will need output when in MakeNNBackwardNode, but as we know, backwards calculation didn't depend the output value of fp node, only depend the output grad.

@tqchen

Can be reproduced on ubuntu 14, with gcc 4.9.2

src/./op_util.h:98:82: error: converting to ‘std::unordered_map<std::basic_string, std::basic_string >’ from initializer list would use explicit constructor ‘std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::unordered_map(std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::size_type, const hasher&, const key_equal&, const allocator_type&) [with _Key = std::basic_string; _Tp = std::basic_string; _Hash = std::hash<std::basic_string >; _Pred = std::equal_to<std::basic_string >; _Alloc = std::allocator<std::pair<const std::basic_string, std::basic_string > >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::size_type = long unsigned int; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::hasher = std::hash<std::basic_string >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::key_equal = std::equal_to<std::basic_string >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::allocator_type = std::allocator<std::pair<const std::basic_string, std::basic_string > >]’
std::unordered_map<std::string, std::string> kwarg = {}) {
^
src/./op_util.h:115:60: error: converting to ‘std::unordered_map<std::basic_string, std::basic_string >’ from initializer list would use explicit constructor ‘std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::unordered_map(std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::size_type, const hasher&, const key_equal&, const allocator_type&) [with _Key = std::basic_string; _Tp = std::basic_string; _Hash = std::hash<std::basic_string >; _Pred = std::equal_to<std::basic_string >; _Alloc = std::allocator<std::pair<const std::basic_string, std::basic_string > >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::size_type = long unsigned int; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::hasher = std::hash<std::basic_string >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::key_equal = std::equal_to<std::basic_string >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::allocator_type = std::allocator<std::pair<const std::basic_string, std::basic_string > >]’
std::unordered_map<std::string, std::string> kwarg = {}) {
^
src/op_nn.cc: In lambda function:
src/op_nn.cc:352:57: error: converting to ‘std::unordered_map<std::basic_string, std::basic_string >’ from initializer list would use explicit constructor ‘std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::unordered_map(std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::size_type, const hasher&, const key_equal&, const allocator_type&) [with _Key = std::basic_string; _Tp = std::basic_string; _Hash = std::hash<std::basic_string >; _Pred = std::equal_to<std::basic_string >; _Alloc = std::allocator<std::pair<const std::basic_string, std::basic_string > >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::size_type = long unsigned int; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::hasher = std::hash<std::basic_string >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::key_equal = std::equal_to<std::basic_string >; std::unordered_map<_Key, _Tp, _Hash, _Pred, _Alloc>::allocator_type = std::allocator<std::pair<const std::basic_string, std::basic_string > >]’
{ograds[0], n->inputs[0]});

Here's the error I am getting :

g++ -std=c++11 -std=c++11 -Wall -O2 -msse2  -Wno-unknown-pragmas -funroll-loops -fPIC -Iinclude -Idmlc-core/include -I/home/kapil/Documents/my-nn-library/tinyflow/nnvm/include -I/home/kapil/torch/install/include -I/home/kapil/torch/install/include/TH -I/home/kapil/torch/install/include/THC/ -MM -MT build/src/session.o src/session.cc >build/src/session.d
g++ -std=c++11 -c -std=c++11 -Wall -O2 -msse2  -Wno-unknown-pragmas -funroll-loops -fPIC -Iinclude -Idmlc-core/include -I/home/kapil/Documents/my-nn-library/tinyflow/nnvm/include -I/home/kapil/torch/install/include -I/home/kapil/torch/install/include/TH -I/home/kapil/torch/install/include/THC/ -c src/session.cc -o build/src/session.o
In file included from src/./torch/torch_util.h:10:0,
                 from src/session.cc:11:
dmlc-core/include/dmlc/lua.h: In member function ‘void dmlc::LuaRef::SetByPopStack_(dmlc::LuaState*)’:
dmlc-core/include/dmlc/lua.h:482:40: error: ‘lua_ref’ was not declared in this scope
     ref_ = lua_ref(L, LUA_REGISTRYINDEX);
                                        ^
In file included from dmlc-core/include/dmlc/lua.h:34:0,
                 from src/./torch/torch_util.h:10,
                 from src/session.cc:11:
/home/kapil/torch/install/include/luaT.h: At global scope:
/home/kapil/torch/install/include/luaT.h:41:12: warning: ‘int luaL_typerror(lua_State*, int, const char*)’ defined but not used [-Wunused-function]
 static int luaL_typerror(lua_State *L, int narg, const char *tname)
            ^
Makefile:55: recipe for target 'build/src/session.o' failed
make: *** [build/src/session.o] Error 1

The error points to dmlc-core code in here. I tried to compile dmlc-core using cmake and it builds successfully. I dont know why the tinyflow makefile is giving the error.

Hello,
my torch is installed correct(can run) and tinyflow is build ok, libtinyflow.so is in ./lib/.

when run example with this cmd:

export LD_LIBRARY_PATH=/home/work/wuwei/project/torch/install/lib:${LD_LIBRARY_PATH}
export PYTHONPATH=${PYTHONPATH}:/home/work/wuwei/project/dmlc/tinyflow/python:/home/work/wuwei/project/dmlc/nnvm/python
python -u mnist_softmax.py

it will output the error log:

Traceback (most recent call last):
  File "mnist_softmax.py", line 5, in <module>
    import tinyflow as tf
  File "/home/work/wuwei/project/dmlc/tinyflow/python/tinyflow/__init__.py", line 3, in <module>
    from . import _base
  File "/home/work/wuwei/project/dmlc/tinyflow/python/tinyflow/_base.py", line 24, in <module>
    from nnvm.name import NameManager
  File "/home/work/wuwei/project/dmlc/nnvm/python/nnvm/__init__.py", line 6, in <module>
    from . import _base
  File "/home/work/wuwei/project/dmlc/nnvm/python/nnvm/_base.py", line 43, in <module>
    _LIB = _load_lib()
  File "/home/work/wuwei/project/dmlc/nnvm/python/nnvm/_base.py", line 35, in _load_lib
    lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_GLOBAL)
  File "/usr/lib64/python2.7/ctypes/__init__.py", line 360, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: /home/work/wuwei/project/torch/install/lib/libTHC.so: undefined symbol: THC_float2half

it should be the libTHC.so can not found by tinyflow, but i had add it to the environment and this file is also exist.
and i found when using ldd libs/lib/libtinyflow.so, log is

    linux-vdso.so.1 =>  (0x00007fff415fe000)
    /lib64/libharfbuzz.so (0x00007fc8044e0000)
    libluajit.so => /home/work/wuwei/project/torch/install/lib/libluajit.so (0x00007fc804246000)
    libluaT.so => /home/work/wuwei/project/torch/install/lib/libluaT.so (0x00007fc80403d000)
    libTH.so => /home/work/wuwei/project/torch/install/lib/libTH.so (0x00007fc803bd6000)
    libTHC.so => /home/work/wuwei/project/torch/install/lib/libTHC.so (0x00007fc801cc4000)
    libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007fc8019bc000)
    libm.so.6 => /lib64/libm.so.6 (0x00007fc8016b9000)
    libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fc8014a3000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fc801287000)
    libc.so.6 => /lib64/libc.so.6 (0x00007fc800ec5000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fc804a77000)
    libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007fc800b8e000)
    libfreetype.so.6 => /lib64/libfreetype.so.6 (0x00007fc8008e8000)
    libgraphite2.so.3 => /lib64/libgraphite2.so.3 (0x00007fc8006cb000)
    libreadline.so.6 => /lib64/libreadline.so.6 (0x00007fc800485000)
    libdl.so.2 => /lib64/libdl.so.2 (0x00007fc800281000)
    libgomp.so.1 => /lib64/libgomp.so.1 (0x00007fc800069000)
    libopenblas.so.0 => /opt/OpenBLAS/lib/libopenblas.so.0 (0x00007fc7ff037000)
    librt.so.1 => /lib64/librt.so.1 (0x00007fc7fee2e000)
    libcublas.so.7.5 => /usr/local/cuda/lib64/libcublas.so.7.5 (0x00007fc7fd54f000)
    libcurand.so.7.5 => /usr/local/cuda/lib64/libcurand.so.7.5 (0x00007fc7f9ce7000)
    libtinfo.so.5 => /lib64/libtinfo.so.5 (0x00007fc7f9abc000)
    libgfortran.so.3 => /lib64/libgfortran.so.3 (0x00007fc7f979a000)
    libquadmath.so.0 => /lib64/libquadmath.so.0 (0x00007fc7f955d000)

so /home/work/wuwei/project/torch/install/lib/libTHC.so is linked.

There is lua code _matmul_bacward registered for gradient calculation for matmul. But in nn_module, there is MakeNNBackwardNode only registered for gradient node generation, where is the gradient calculation(like _conv_backward) for that node, and what is the purpose to generate graident node in op_nn?