zhanghang1989 / pytorch-encoding Goto Github PK

View Code? Open in Web Editor NEW

2.0K 43.0 450.0 21.21 MB

A CV toolkit for my papers.

Home Page: https://hangzhang.org/PyTorch-Encoding/

License: MIT License

Python 75.51% Makefile 0.04% C++ 9.23% Cuda 14.62% Shell 0.48% Dockerfile 0.12%

deep-learning batchnorm synchronized-batchnorm encoding-layer

pytorch-encoding's Introduction

PyTorch-Encoding

created by Hang Zhang

Documentation

Please visit the Docs for detail instructions of installation and usage.
Please visit the link to image classification models.
Please visit the link to semantic segmentation models.

Citations

ResNeSt: Split-Attention Networks [arXiv]
Hang Zhang, Chongruo Wu, Zhongyue Zhang, Yi Zhu, Zhi Zhang, Haibin Lin, Yue Sun, Tong He, Jonas Muller, R. Manmatha, Mu Li and Alex Smola

@article{zhang2020resnest,
title={ResNeSt: Split-Attention Networks},
author={Zhang, Hang and Wu, Chongruo and Zhang, Zhongyue and Zhu, Yi and Zhang, Zhi and Lin, Haibin and Sun, Yue and He, Tong and Muller, Jonas and Manmatha, R. and Li, Mu and Smola, Alexander},
journal={arXiv preprint},
year={2020}
}

Context Encoding for Semantic Segmentation [arXiv]
Hang Zhang, Kristin Dana, Jianping Shi, Zhongyue Zhang, Xiaogang Wang, Ambrish Tyagi, Amit Agrawal

@InProceedings{Zhang_2018_CVPR,
author = {Zhang, Hang and Dana, Kristin and Shi, Jianping and Zhang, Zhongyue and Wang, Xiaogang and Tyagi, Ambrish and Agrawal, Amit},
title = {Context Encoding for Semantic Segmentation},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2018}
}

Deep TEN: Texture Encoding Network [arXiv]
Hang Zhang, Jia Xue, Kristin Dana

@InProceedings{Zhang_2017_CVPR,
author = {Zhang, Hang and Xue, Jia and Dana, Kristin},
title = {Deep TEN: Texture Encoding Network},
booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {July},
year = {2017}
}

pytorch-encoding's People

Contributors

Stargazers

Watchers

Forkers

keyky zoonono wangjingbo1219 ucasqcz ml-lab zijundeng david2021 amoliu speedinghzl fengqian1989 caomw computervisionrutgers philipppelz boundles maokeai negative09 lfz marvinteichmann lexie0212 huaxinxiao baby47 wuzzh chelovek21 mahlermozart haptics-nri shubhampachori12110095 grow-yhq leeesangwon jacksu20160407 hdjang dongzhuoyao locussam donnyyou wenhuach stevengrove damax18 mygit007hub wenqingchu suyanzhou626 kenstler sud0301 yanwei-li yiwenshaostephen afcarl dingke joseph-zhong jiangqh bjchen666 gengshan-y undercontroller jerryzcn hanxueming126 dearleiii oujieww 94mia zhengsx happog dl-85 wpfhtl yu45020 wdddddd dongdongbai dukebw ricardozzf liu3xing3long jjsjann123 xychenunc mengxi-ascent linbooo deepylt wutianyirosun tengsz winwinjjiang lxtgh blankworld arunkumarramanan holyhao codeaudit wwxfromtju xiaopingzeng xjtueducation holyseven alialamiidrissi algohunt codes-kzhan tianhaijie summer1719 yxecho115 zhenyuczy zilongzhong jeremy-yung mhaiyang supermousse cyhbrilliant stjordanis britney-f andyweizhao bnu-wangxun xdcesc plyang

pytorch-encoding's Issues

RuntimeError device-side assert triggered

Hi @zhanghang1989,
I got the following error when run: python3 main.py --dataset minc --model deepten --batch-size 64 --lr 0.01 --epochs 60

THCudaCheck FAIL file=/pytorch/torch/lib/THC/generic/THCStorage.cu line=58 error=59 : device-side assert triggered
Traceback (most recent call last):
File "main.py", line 171, in
main()
File "main.py", line 157, in main
train(epoch)
File "main.py", line 92, in train
loss.backward()
File "/usr/local/lib/python3.6/site-packages/torch/autograd/variable.py", line 167, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
File "/usr/local/lib/python3.6/site-packages/torch/autograd/init.py", line 99, in backward
variables, grad_variables, retain_graph)
RuntimeError: cuda runtime error (59) : device-side assert triggered at /pytorch/torch/lib/THC/generic/THCStorage.cu:58

/home/PyTorch-Encoding-Layer-master/encoding/src/encoding_lib.cpp:12:26: fatal error: thc_encoding.h: No such file or directory

[Python 3.5] Installation fails - Workaround avaible

When running 'python setup.py install' on my machine I get the error message below.

The error indicates, that the code of test/test.py fails. The reason for this is, that test loads the local module '.encoding' instead of the installed one. A possible workaround is to run python setup.py build_ext --inplace before running python setup.py install. This creates a compiled lib inside the local module.

I assume that the author has run the former command at some point, so that he does not run into the issue. I think however, that this should ether be documented [hacky solution], of fixed in the sense that test.py should load the installed module and not the local one.

<< python setup.py install
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
...
...
creating encoding.egg-info
writing encoding.egg-info/PKG-INFO
...
removing '/home/mifs/mttt2/.virtualenvs/pytorch/lib/python3.5/site-packages/encoding-0.1.0+01946d4-py3.5.egg-info' (and everything under it)
Copying encoding.egg-info to /home/mifs/mttt2/.virtualenvs/pytorch/lib/python3.5/site-packages/encoding-0.1.0+01946d4-py3.5.egg-info
running install_scripts
Traceback (most recent call last):
  File "test/test.py", line 11, in <module>
    import encoding
  File "/home/mifs/mttt2/workarea/github/PyTorch-Encoding/encoding/__init__.py", line 13, in <module>
    import encoding.nn
  File "/home/mifs/mttt2/workarea/github/PyTorch-Encoding/encoding/nn/__init__.py", line 11, in <module>
    from .encoding import *
  File "/home/mifs/mttt2/workarea/github/PyTorch-Encoding/encoding/nn/encoding.py", line 17, in <module>
    from .._ext import encoding_lib
  File "/home/mifs/mttt2/workarea/github/PyTorch-Encoding/encoding/_ext/encoding_lib/__init__.py", line 3, in <module>
    from ._encoding_lib import lib as _lib, ffi as _ffi
ImportError: No module named 'encoding._ext.encoding_lib._encoding_lib'
Traceback (most recent call last):
  File "setup.py", line 66, in <module>
    'install': install,
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "setup.py", line 26, in run
    subprocess.check_call("python test/test.py".split())
  File "/usr/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'test/test.py']' returned non-zero exit status 1

loss just not converge when adding a L2 norm layer after Encoding layer

Hi, I am very interested in your great work. I try to make use of Encoding Layer in my own dataset, but I found that training loss just not decrease(so so slow) when L2 norm layer is used after Encoding Layer.

Could you please tell me what role the L2 norm plays in your experiments and whether it fluences the performance or not?

python setup.py install PROBLEM

Hi,
I failed to install from this source .Why is it?
`
/usr/include/c++/4.8/cmath(406): error: expected a ";"

/usr/include/c++/4.8/cmath(443): error: inline specifier allowed on function declarations only

/usr/include/c++/4.8/cmath(443): error: variable "std::constexpr" has already been defined

/usr/include/c++/4.8/cmath(443): error: expected a ";"

Error limit reached.
100 errors detected in the compilation of "/tmp/tmpxft_00002889_00000000-17_thc_encoding.compute_61.cpp1.ii".
Compilation terminated.
CMake Error at ENCODING_generated_thc_encoding.cu.o.cmake:264 (message):
Error generating file
/home/bbbian/local/PyTorch-Encoding/encoding/lib/CMakeFiles/ENCODING.dir/kernel/./ENCODING_generated_thc_encoding.cu.o

make[2]: *** [CMakeFiles/ENCODING.dir/kernel/./ENCODING_generated_thc_encoding.cu.o] Error 1
make[1]: *** [CMakeFiles/ENCODING.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
File "setup.py", line 66, in
'install': install,
File "/home/bbbian/local/anaconda3/lib/python3.6/distutils/core.py", line 108, in setup
_setup_distribution = dist = klass(attrs)
File "/home/bbbian/local/anaconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/dist.py", line 318, in init
File "/home/bbbian/local/anaconda3/lib/python3.6/distutils/dist.py", line 281, in init
self.finalize_options()
File "/home/bbbian/local/anaconda3/lib/python3.6/site-packages/setuptools-27.2.0-py3.6.egg/setuptools/dist.py", line 375, in finalize_options
File "/home/bbbian/local/anaconda3/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 187, in cffi_modules
add_cffi_module(dist, cffi_module)
File "/home/bbbian/local/anaconda3/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
execfile(build_file_name, mod_vars)
File "/home/bbbian/local/anaconda3/lib/python3.6/site-packages/cffi/setuptools_ext.py", line 25, in execfile
exec(code, glob, glob)
File "/home/bbbian/local/PyTorch-Encoding/build.py", line 37, in
subprocess.check_call(build_all_cmd, env=dict(os.environ))
File "/home/bbbian/local/anaconda3/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bash', 'encoding/make.sh']' returned non-zero exit status 2.
`

problem when complie

Hi, hang,
when I compile the code, it shows the following error. Could you help to check where is wrong?

Some problems during installation

Hi @zhanghang1989, thanks for your sharing! When installing the Encoding package, I encounter some problems and solve these problems.

thc_encoding.h: No such file or directory #8
the thc_encoding.h is stored in encoding/kernel, so I just add include path in build.py
os.path.join(this_file,'encoding/src/'), os.path.join(this_file,'encoding/kernel/')]
THCDeviceTensor.cuh: No such file or directory
git clone https://github.com/zhanghang1989/PyTorch-Encoding & git checkout v0.2.0.
I switch pytorch branch from master to v0.2, because in master branch, there is no such file.
SyntaxError: invalid syntax in File "/home/hzl/anaconda2/lib/python2.7/site-packages/encoding/parallel.py", line 232
return (modules[0](*inputs[0], *targets[0], **kwargs_tup[0]), )
I just remove *before inputs[0]
test fail in test_dilated_avgpool
$$ I just ignore it.

Finally, I want to use the synchronized BatchNorm, could you please provide some example code to show how to use it?
Thanks again!

Help needed with understanding the code

I am new to github. I am trying to understand the code particularly where the encoding part is done. I am unclear where 'Encoding_Float_scaledl2_forward' is implemented. Could you please guide me to that part?

libENCODING.so library missing

Hello,

I've tried to install this Encoding-Layer on Linux 16.04 but I have the following error occuring during the installation :

x86_64-linux-gnu-g++: error: /usr/local/lib/python3.5/dist-packages/torch/lib/libENCODING.so: No file or repository of this type
error: command 'x86_64-linux-gnu-g++' failed with exit status 1

I tried to find this library to see if the problem was coming from a wrong path but it looks like this library doesn't exist on my computer.
I have installed Pytorch from the source following the instructions with no problems and I'm also using Cuda 8.0.
Could you help me to find a solution to this problem please ?

Thanks

AttributeError: module 'encoding._ext.encoding_lib' has no attribute 'Encoding_Float_scaledl2_forward'

Hi Z-Hang:

When i test EncNet on CIFAR-10 using :
python main.py --dataset cifar10 --model encnetdrop --widen 8 --ncodes 32 --resume model/encnet_cifar.pth.tar --eval

I got the following error.Could you help me solve it.

(python setup.py install has done and successed. )

my_env: pytorch0.4,anaconda3,ubuntu16.04

Using step LR Scheduler!
:   0%|                                                                          | 0/79 [00:00<?, ?it/s]Traceback (most recent call last):
  File "main.py", line 172, in <module>
    main()
  File "main.py", line 154, in main
    test(args.start_epoch)
  File "main.py", line 116, in test
    output = model(data)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 112, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/newP/En/PyTorch-Encoding/experiments/recognition/model/encnetdrop.py", line 59, in forward
    return self.model(input)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
    input = module(input)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
    input = module(input)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/newP/En/PyTorch-Encoding/experiments/recognition/model/mynn.py", line 157, in forward
    out = self.se(out)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/newP/En/PyTorch-Encoding/experiments/recognition/model/mynn.py", line 127, in forward
    y = self.fc(x).view(b, c, 1, 1)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/container.py", line 91, in forward
    input = module(input)
  File "/root/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/root/anaconda3/lib/python3.6/site-packages/encoding/nn/encoding.py", line 151, in forward
    A = F.softmax(scaledL2(X, self.codewords, self.scale), dim=1)
  File "/root/anaconda3/lib/python3.6/site-packages/encoding/functions/encoding.py", line 139, in scaledL2
    return _scaledL2.apply(X, C, S)
  File "/root/anaconda3/lib/python3.6/site-packages/encoding/functions/encoding.py", line 93, in forward
    encoding_lib.Encoding_Float_scaledl2_forward(SL, X, C, S)
AttributeError: module 'encoding._ext.encoding_lib' has no attribute 'Encoding_Float_scaledl2_forward'

Error while installing Pytorch encoding

I get the following error message
File "test/test.py", line 11, in
import encoding
File "/home/turing/anaconda2/lib/python2.7/site-packages/encoding/init.py", line 13, in
import encoding.nn
File "/home/turing/anaconda2/lib/python2.7/site-packages/encoding/nn/init.py", line 11, in
from .encoding import *
File "/home/turing/anaconda2/lib/python2.7/site-packages/encoding/nn/encoding.py", line 19, in
from ..parallel import my_data_parallel
File "/home/turing/anaconda2/lib/python2.7/site-packages/encoding/parallel.py", line 259
return (modules[0](*inputs[0], *targets[0], **kwargs_tup[0]), )

Compile Error

Hi, hang,
I follow the instruction on http://hangzh.com/PyTorch-Encoding/notes/compile.html. But it shows the following error when I run "python setup.py install". Could you help to check where is wrong?
Thanks a lot.

error when install encoding package

hi, I am now working on a segmentation project so I do need this package to solve the batch norm and uneven data parallel problems. However, I encountered this error when I installed the package in python3.5 env:
Testing scaledL2(): True
Traceback (most recent call last):
File "test/test.py", line 88, in
test_encoding()
File "test/test.py", line 52, in test_encoding
test = gradcheck(layer, input, eps=1e-6, atol=1e-4)
File "/scratch/jindi/.condaEnvs/python3/lib/python3.5/site-packages/torch/autograd/gradcheck.py", line 154, in gradcheck
output = func(*inputs)
File "/scratch/jindi/.condaEnvs/python3/lib/python3.5/site-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "/scratch/jindi/.condaEnvs/python3/lib/python3.5/site-packages/encoding/nn/encoding.py", line 97, in forward
A = F.softmax(scaledL2(X, self.codewords, self.scale), dim=1)
TypeError: softmax() got an unexpected keyword argument 'dim'
Could you help solve this installment problem? Thanks!

install error: No module named 'encoding._ext.encoding_lib._encoding_lib'

Hi, I follow the instructions given: fisrt install pytorch from source and then install Encoding, but i met a error: 'No module named 'encoding._ext.encoding_lib._encoding_lib'. I install pytorch in python3.6.

Do you have any suggestions? Thanks.

Training on new dataset

Thanks for the Code. How can I train the network on my own dataset of images ?

Dropbox-error

I can't download minc.pth.tar. Its link needs to be update.

Issue with syncbn

Thank you for your effort. I got an error while running a model in syncbn.py. I don't know have any clue about solving this. Can you help me out. Here is the error:
Traceback (most recent call last):
loss.backward()
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/autograd/variable.py", line 167, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/torch/autograd/init.py", line 99, in backward
variables, grad_variables, retain_graph)
File "/home/ubuntu/anaconda3/lib/python3.6/site-packages/encoding/functions/syncbn.py", line 102, in backward
self.training)
NameError: name 'self' is not defined

Attribute Error

Hello

Thanks for your kind code sharing.
I have one problem while using your code.

I defined input variable and encoding layer as below.

X = Variable(torch.cuda.DoubleTensor(B,C,H,W).uniform_(-0.5,0.5), requires_grad=True)
layer = encoding.Encoding(C,K).double().cuda()

However, I receive an error when I attempt to forward the input.

E = layer(X)
AttributeError: module 'encoding._ext.encoding_lib' has no attribute 'Encoding_Double_scaledl2_forward'

Any suggestions?

Thanks

c99 flag error

Hi Zhang Hang, thank you for your effort and releasing this code.

I have insalled pytorch from source and use python 3.5 (anaconda3). When I run python setup.py install I got the following error:

creating build/temp.linux-x86_64-3.5/data/home/liao/Downloads/PyTorch-Encoding [3/1814]
creating build/temp.linux-x86_64-3.5/data/home/liao/Downloads/PyTorch-Encoding/encoding
creating build/temp.linux-x86_64-3.5/data/home/liao/Downloads/PyTorch-Encoding/encoding/src
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -fPIC -DWITH_CUDA -I/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../
../lib/include -I/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH -I/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/
ffi/../../lib/include/THC -I/usr/local/cuda/include -I/home/liao/anaconda3/lib/python3.5/site-packages/torch/lib/include -I/data/home/liao/Downloads/PyTorch-Encoding/
encoding/kernel -I/data/home/liao/Downloads/PyTorch-Encoding/encoding/kernel/include -I/data/home/liao/Downloads/PyTorch-Encoding/encoding/src/ -I/home/liao/anaconda3
/include/python3.5m -c build/temp.linux-x86_64-3.5/encoding._ext.encoding_lib._encoding_lib.c -o build/temp.linux-x86_64-3.5/build/temp.linux-x86_64-3.5/encoding._ext
.encoding_lib._encoding_lib.o
In file included from /home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THVector.h:5:0,
from /home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/TH.h:12,
from build/temp.linux-x86_64-3.5/encoding._ext.encoding_lib._encoding_lib.c:433:
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: In function ‘TH_digamma’:
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:139:3: error: ‘for’ loop initial declarations are only allowed in C99 o
r C11 mode
for (int i = 0; i < 4; ++i) {
^
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:139:3: note: use option -std=c99, -std=gnu99, -std=c11 or -std=gnu11 to
compile your code
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: In function ‘TH_digammaf’:
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:154:3: error: ‘for’ loop initial declarations are only allowed in C99 o
r C11 mode
for (int i = 0; i < 4; ++i) {
^
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: In function ‘TH_trigamma’:
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:172:3: error: ‘for’ loop initial declarations are only allowed in C99 o
r C11 mode
for (int i = 0; i < 6; ++i) {
^
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h: In function ‘TH_trigammaf’:
/home/liao/anaconda3/lib/python3.5/site-packages/torch/utils/ffi/../../lib/include/TH/THMath.h:190:3: error: ‘for’ loop initial declarations are only allowed in C99 o
r C11 mode
for (int i = 0; i < 6; ++i) {

error: command 'gcc' failed with exit status 1

I know that I should add -std=c99 somewhere but I don't know where to add. I have tried to modify the CmakeLists.txt
48 SET(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -std=c99")
but it still does not work

Could you help me solving this? Thank you

How to use the DataParallelCriterion ,DataParallelModel

I fllow the insturction in this page in my code.
However I got this error.
Traceback (most recent call last):
File "train_psp_resnet101.py", line 232, in
train(cfg)
File "train_psp_resnet101.py", line 124, in train
loss = loss_fn(outputs, labels)
File "/home/xiangtai/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 491, in call
result = self.forward(input, **kwargs)
File "/home/xiangtai/anaconda3/lib/python3.6/site-packages/encoding/parallel.py", line 134, in forward
outputs = _criterion_parallel_apply(replicas, inputs, targets, kwargs)
File "/home/xiangtai/anaconda3/lib/python3.6/site-packages/encoding/parallel.py", line 188, in _criterion_parallel_apply
raise output
File "/home/xiangtai/anaconda3/lib/python3.6/site-packages/encoding/parallel.py", line 163, in _worker
output = module((input + target), **kwargs)
TypeError: add() received an invalid combination of arguments - got (tuple), but expected one of:

(Tensor other, float alpha)
(float other, float alpha)

the outputs is a list,each item is a small tensor(each tensor is on different gpu), labels is not a list, but a tensor on gpu:0
@zhanghang1989

train example of syncbn

Hi, could you provide an example of train syncbn?

libTHC.so.1: cannot open shared object file: No such file or directory

Hi,
I installed the pytorch from source but it can not find libTHC.so.1. However, it is in my virtual env since I checked with locate. /home/ubuntu/pyramid/lib/python2.7/site-packages/torch/lib/libTHC.so.1
It seems that libTHC.so.1 is missing.

running install_scripts
Traceback (most recent call last):
File "test/test.py", line 11, in
import encoding
File "/home/ubuntu/pyramid/local/lib/python2.7/site-packages/encoding/init.py", line 13, in
import encoding.nn
File "/home/ubuntu/pyramid/local/lib/python2.7/site-packages/encoding/nn/init.py", line 11, in
from .syncbn import *
File "/home/ubuntu/pyramid/local/lib/python2.7/site-packages/encoding/nn/syncbn.py", line 23, in
from ..functions import view_each, multi_each, sum_each, batchnormtrain, batchnormeval, sum_square
File "/home/ubuntu/pyramid/local/lib/python2.7/site-packages/encoding/functions/init.py", line 2, in
from .encoding import *
File "/home/ubuntu/pyramid/local/lib/python2.7/site-packages/encoding/functions/encoding.py", line 16, in
from .._ext import encoding_lib
File "/home/ubuntu/pyramid/local/lib/python2.7/site-packages/encoding/_ext/encoding_lib/init.py", line 3, in
from ._encoding_lib import lib as _lib, ffi as _ffi
ImportError: libTHC.so.1: cannot open shared object file: No such file or directory

No module named 'encoding._ext.encoding_lib._encoding_lib'

HI,

I try to run your project. I can run it with terminal, however when I try to debug with pycharm, it lack of lib:

Traceback (most recent call last):
File "/media/code/PyTorch-Encoding/experiments/main.py", line 23, in
from encoding.utils import *
File "/media/code/PyTorch-Encoding/encoding/init.py", line 13, in
import encoding.nn
File "/media/code/PyTorch-Encoding/encoding/nn/init.py", line 11, in
from .syncbn import *
File "/media/code/PyTorch-Encoding/encoding/nn/syncbn.py", line 23, in
from ..functions import view_each, multi_each, sum_each, batchnormtrain, batchnormeval, sum_square
File "/media/code/PyTorch-Encoding/encoding/functions/init.py", line 2, in
from .encoding import *
File "/media/code/PyTorch-Encoding/encoding/functions/encoding.py", line 16, in
from .._ext import encoding_lib
File "/media/code/PyTorch-Encoding/encoding/_ext/encoding_lib/init.py", line 3, in
from ._encoding_lib import lib as _lib, ffi as _ffi
ImportError: No module named 'encoding._ext.encoding_lib._encoding_lib'

In folder encoding_lib do not have _encoding_lib but when I run with terminal it still can run. Maybe you will know the reason. Please help me.

Thanks

Install Issue

Hello, I was able to install the library but for some reason the test_dilated_avgpool test case fails with the error RuntimeError: there are no graph nodes that require computing gradients. Is this anticipated?

CriterionDataParallel example

Can you give an example of how to use CriterionDataParallel?
Thanks

model.eval() seems not work well

It's very kind of your work ! There's still something i want for your help!
My code for model and criterion parallel like this:
model = encoding.parallel.ModelDataParallel(model,device_ids=[0,1,2])
criterion = encoding.parallel.CriterionDataParallel(criterion,device_ids=[0,1,2])
Training process is going well, but when i turn to model.eval(), i got explosion of loss.

Is there something wrong with model.eval()?

model.encodingnet

i try to use the DeepTen.
where is module 'model.encodingnet' ?

File "", line 985, in _gcd_import
File "", line 968, in _find_and_load
File "", line 955, in _find_and_load_unlocked
ImportError: No module named 'model.encodingnet'

syn-batchnorm : error during validation, train seems good.

File "~/.local/lib/python3.6/site-packages/encoding/nn/syncbn.py", line 41, in forward
std = (self.running_var.clamp(self.eps)).sqrt()
TypeError: clamp received an invalid combination of arguments - got (float), but expected one of:

(float min)
didn't match because some of the arguments have invalid types: (float)
(float max)
didn't match because some of the arguments have invalid types: (float)
(float min, float max)

My env: centos 7, python3.6, pytorch 0.3.1, cuda-8.0

Thanks in advance.

Cuda memory leak in forward process

In the forward process, the Cuda memory increase by about 10MB (one syncbn-2d) for each batch. I have tried the different pytorch version (0.3.0, 0.3.1, 0.4.0), but the issue still occurred.

import torch
from torch.autograd import Variable
from torch.nn.parallel.data_parallel import DataParallel
from encoding.nn import BatchNorm2d

net = BatchNorm2d(128).cuda()
net = DataParallel(net, device_ids=list(range(8)))

while True:
x = Variable(torch.randn(32, 128, 16, 16), requires_grad=True)
net(x)
loss = x.sum()
loss.backward()

tuple problem

Thanks for your effort.I want to use your "BN" and "SelfDataParallel", but i got a troube.can you help me
out = self.conv1(x)
File "/home/hdc/anaconda3/envs/pytorch_gpu_3.6/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/home/hdc/anaconda3/envs/pytorch_gpu_3.6/lib/python3.6/site-packages/torch/nn/modules/conv.py", line 388, in forward
self.padding, self.dilation, self.groups)
File "/home/hdc/anaconda3/envs/pytorch_gpu_3.6/lib/python3.6/site-packages/torch/nn/functional.py", line 120, in conv3d
if input is not None and input.dim() != 5:
AttributeError: 'tuple' object has no attribute 'dim'

Encoding folder

Hi,

Thank for your project. I have confuse with encoding folder in your project.
When I build setup.py, it run test.py in test folder and it will run function in encoding folder, but encoding folder in python3.5. It will have error and I do not understand why your project have encoding folder but it do not install in it:
Testing scaledL2(): True
Traceback (most recent call last):
File "test/test.py", line 88, in
test_encoding()
File "test/test.py", line 52, in test_encoding
test = gradcheck(layer, input, eps=1e-6, atol=1e-4)
File "/usr/local/lib/python3.5/dist-packages/torch/autograd/gradcheck.py", line 154, in gradcheck
output = func(*inputs)
File "/usr/local/lib/python3.5/dist-packages/torch/nn/modules/module.py", line 224, in call
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/encoding/nn/encoding.py", line 97, in forward
A = F.softmax(scaledL2(X, self.codewords, self.scale), dim=2)
TypeError: softmax() got an unexpected keyword argument 'dim'

Thanks

compile error with pytorch0.4

Hi Hang:
When i compile encoding using "python setup.py install" , i got the following error.Could you help me solve it. Is encoding/syncbn compatible with the latest pytorch0.4?
my environment: pytorch0.4,anaconda3,ubuntu16.04

module 'encoding._ext.encoding_lib' has no attribute 'Encoding_Float_sum_square_Forward'

Although compiled successfully (carefully following the instructions), or at least no error messages has been shown during compilation, it seems that the module is not installed correctly.

I get the following error message:


  File "/home/<user>/code/networks/extractors.py", line 259, in forward
    out = self.start_features(x)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/nn/modules/container.py", line 67, in forward
    input = module(input)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/nn/modules/module.py", line 357, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/encoding/nn/syncbn.py", line 43, in forward
    xsum, xsquare = self.sharedT(sum_square(input.unsqueeze(3)))
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/encoding/functions/syncbn.py", line 59, in sum_square
    return _sum_square.apply(input)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/encoding/functions/syncbn.py", line 28, in forward
    encoding_lib.Encoding_Float_sum_square_Forward(
AttributeError: module 'encoding._ext.encoding_lib' has no attribute 'Encoding_Float_sum_square_Forward'

extractor.py line 259 refers to extractor.py line 250 from a deep learning repo where I replaced nn.BatchNorm2d with encoding.nn.BatchNorm2d.

Indeed, the folder /home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/encoding/_ext/encoding_lib does not contain any .py files for me.

What could be the issue here?

I am using:
pytorch 0.3.1 (build from source), I cloned the pytorch repo at state 2b4748011b5881583567bb166801ca6625f2fdda which is associated with the 0.3.1 release. pytorch 0.4 is too unstable and has too many (partly unlisted) breaking changes. I had to revert to v0.3.1
conda 4.3 / cuda 8.0 / Edit: python 3.5

No module named _encoding_lib when running python setup.py install

Traceback (most recent call last):
File "test/test.py", line 11, in
import encoding
File "/home/work/Library/PyTorch-Encoding/encoding/init.py", line 13, in
import encoding.nn
File "/home/work/Library/PyTorch-Encoding/encoding/nn/init.py", line 11, in
from .encoding import *
File "/home/work/Library/PyTorch-Encoding/encoding/nn/encoding.py", line 17, in
from .._ext import encoding_lib
File "/home/work/Library/PyTorch-Encoding/encoding/_ext/encoding_lib/init.py", line 3, in
from ._encoding_lib import lib as _lib, ffi as _ffi
ImportError: No module named _encoding_lib

Runtime Error

python main.py --dataset minc --model deepten --batch-size 64 --lr 0.01 --epochs 60

/media/data_5t/yc/anaconda3/envs/py36/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py:24: UserWarning:
There is an imbalance between your GPUs. You may want to exclude GPU 1 which
has less than 75% of the memory or cores of GPU 0. You can do so by setting
the device_ids argument to DataParallel, or by setting the CUDA_VISIBLE_DEVICES
environment variable.
warnings.warn(imbalance_warn.format(device_ids[min_pos], device_ids[max_pos]))
Using step LR Scheduler!
: 0%| | 0/764 [00:00<?, ?it/s]
=>Epoches 1, learning rate = 0.0100, previous best = 100.0000
Traceback (most recent call last):
File "main.py", line 171, in
main()
File "main.py", line 157, in main
train(epoch)
File "main.py", line 92, in train
loss.backward()
File "/media/data_5t/yc/anaconda3/envs/py36/lib/python3.6/site-packages/torch/autograd/variable.py", line 120, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/media/data_5t/yc/anaconda3/envs/py36/lib/python3.6/site-packages/torch/autograd/init.py", line 81, in backward
variables, grad_variables, retain_graph, create_graph)
RuntimeError: cublas runtime error : the GPU program failed to execute at /media/data_5t/yc/pytorch/aten/src/THC/THCBlas.cu:249

my env:
Anaconda, python 3.6
cuda 8.0, pytorch has installed from source and current version is 0.4. torchvision also installs from source (0.2), ubuntu 16.04.

complie errors

Hi, hang,
I visit http://hangzh.com/PyTorch-Encoding/notes/compile.html#install-from-source. And when I use python setup.py install, it shows the following error. Could you help to check where is wrong?

I think this is due to encoding/make.sh, so I execute ./encoding/make.sh, it shows the following error

No rule to make target libATen.so.1 needed by libENCODING.so

Hi Hang,

Thanks for your great work.

When I prepare to compile Pytorch-Encoding, I met an error. Here is a piece of snapshot.

-- Configuring done
-- Generating done
-- Build files have been written to: /home/zhouhy/Library/PyTorch-Encoding/encoding/lib
[ 50%] Building NVCC (Device) object CMakeFiles/ENCODING.dir/kernel/ENCODING_generated_thc_encoding.cu.o
Scanning dependencies of target ENCODING
make[2]: *** No rule to make target /home/zhouhy/anaconda2/lib/python2.7/site-packages/torch/lib/libATen.so.1', needed by libENCODING.so'. Stop.
make[1]: *** [CMakeFiles/ENCODING.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
File "setup.py", line 67, in

I've compiled pytorch 0.4.0 from source.

Looking forward to your reply.

I can't find the code of encnet

I can't find the part of semantic segmentation in this code ,where should i looking for?

Code for Segmentation

When will the code for segmentation be released?

How to synchronized Batch Normalization?

Thanks for your project.
I find the BN code in encoding/init.py, but i don't konw how to use it? Is the code support multi-GPU now?

CMake Error:install TARGETS given no LIBRARY DESTINATION for shared library target

I try to install this package,but get the following error:

-- TORCH_BUILD_DIR: /usr/local/lib/python2.7/dist-packages/torch/lib
-- Autodetected CUDA architecture(s): 3.0
-- Torch_INSTALL_INCLUDE:/usr/local/lib/python2.7/dist-packages/torch/lib/include/usr/local/lib/python2.7/dist-packages/torch/lib/include/TH/usr/local/lib/python2.7/dist-packages/torch/lib/include/THC/home/ubuntu/pytorch/torch/lib/THC
CMake Error at CMakeLists.txt:78 (INSTALL):
install TARGETS given no LIBRARY DESTINATION for shared library target
"ENCODING".

-- Configuring incomplete, errors occurred!
See also "/home/ubuntu/soushi/reg/DeepTEN/encoding/build/CMakeFiles/CMakeOutput.log".
See also "/home/ubuntu/soushi/reg/DeepTEN/encoding/build/CMakeFiles/CMakeError.log".
make: *** No rule to make target 'install'. Stop.
/usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'extra_compile_args'
warnings.warn(msg)

A problem In Installing

I find this problem in installing
I use the latest version pytorch and anaconda3
so how can I fix it
Thank you!

Copying encoding.egg-info to /home/wang/anaconda3/lib/python3.6/site-packages/encoding-0.1.0+c92a7c2-py3.6.egg-info
running install_scripts
Traceback (most recent call last):
File "test/test.py", line 87, in
test_scaledL2()
File "test/test.py", line 42, in test_scaledL2
test = gradcheck(encoding.functions.scaledL2, input, eps=1e-6, atol=1e-4)
File "/home/wang/anaconda3/lib/python3.6/site-packages/torch/autograd/gradcheck.py", line 163, in gradcheck
output = _differentiable_outputs(func(*inputs))
File "/home/wang/anaconda3/lib/python3.6/site-packages/encoding/functions/encoding.py", line 135, in scaledL2
return _scaledL2.apply(X, C, S)
File "/home/wang/anaconda3/lib/python3.6/site-packages/encoding/functions/encoding.py", line 95, in forward
encoding_lib.Encoding_Double_scaledl2_forward(SL, X, C, S)
AttributeError: module 'encoding._ext.encoding_lib' has no attribute 'Encoding_Double_scaledl2_forward'
Traceback (most recent call last):
File "setup.py", line 64, in
'install': install,
File "/home/wang/anaconda3/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/home/wang/anaconda3/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/home/wang/anaconda3/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "setup.py", line 28, in run
subprocess.check_call("python test/test.py".split())
File "/home/wang/anaconda3/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python', 'test/test.py']' returned non-zero exit status 1.

Broken dependencies

@zhanghang1989
Hi,
when I tried to install the pack, I got this error:
/home/PyTorch-Encoding-Layer-master/encoding/src/encoding_lib.cpp:12:26: fatal error: thc_encoding.h: No such file or directory
It seems that there are some broken dependencies in the package, right?
I thought it might be something related to my PyTorch installation, so I installed all the things from the scratch, but the error remained the same.

Runtime error

When I run python setup.py install , I get the following error :cuda runtime error (30) : unknown error at /pytorch/torch/lib/THC/THCGeneral.c:70 at 'install': TestCommand, in setup.py file

Need some help. I don't understand some code.

Due to my poor skill of pytorch. I found this code in your encodenet.py

# Conv_1
model += [nn.Conv2d(3, self.inplanes, kernel_size=3, padding=1),
					nn.BatchNorm2d(self.inplanes),
					nn.ReLU(inplace=True)]
# Residual units
for i in range(4):
	model += [self._residual_unit(block, num_planes[i], num_blocks[i],
						strides[i])]
# Last conv layer
# TODO norm layer, instance norm?
model += [nn.BatchNorm2d(self.inplanes),
			nn.ReLU(inplace=True),
			Encoding(D=512*self.expansion,K=16),
			nn.BatchNorm1d(16),
			nn.ReLU(inplace=True),
			nn2.View(-1, 512*self.expansion*16),
			nn.Linear(512*self.expansion*16, num_classes)]
self.model = nn.Sequential(*model)

What's the input of Encoding(D=512*self.expansion,K=16). I found your encode kernel which only accept 2 tensor(A,R) and where are the X, C and A?
Although R = X - C, I don't find the X - C in your model

Results of sync-bn module seem not correct

I tried one validation code, but it seems not works:

`
import torch
from torch.autograd import Variable
from torch.nn.parallel.data_parallel import DataParallel
from encoding.nn import BatchNorm2d

net = BatchNorm2d(128).cuda()
net_sync = DataParallel(net, device_ids=list(range(8)))

while True:
x = Variable(torch.randn(32, 128, 16, 16).cuda(), requires_grad=True)
r = net(x)
r_sync = net_sync(x)
print(r.ne(r_sync).long().sum())
print(torch.abs(r - r_sync).mean())
loss = x.sum()
loss.backward()
`

Results:
tensor(1.0486e+06, device='cuda:0')
tensor(1.00000e-02 *
1.3687, device='cuda:0')
tensor(1.0486e+06, device='cuda:0')
tensor(1.00000e-02 *
1.3746, device='cuda:0')
tensor(1.0486e+06, device='cuda:0')
tensor(1.00000e-02 *
1.3329, device='cuda:0')
tensor(1.0486e+06, device='cuda:0')
tensor(1.00000e-02 *
1.3601, device='cuda:0')
tensor(1.0486e+06, device='cuda:0')
tensor(1.00000e-02 *
1.4027, device='cuda:0')
tensor(1.0486e+06, device='cuda:0')
tensor(1.00000e-02 *
1.2757, device='cuda:0')
...

Fail to install in Ubuntu 14.04

Hi.
Thanks for your effort.

I fail to install your library on my machine (Ubuntu 14.04, CUDA 8.0, python3 with anaconda).

$ python setup.py install
-- The C compiler identification is GNU 4.9.4
-- The CXX compiler identification is GNU 4.9.4
-- Check for working C compiler: /usr/bin/gcc
-- Check for working C compiler: /usr/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/gcc
-- Check for working CXX compiler: /usr/bin/gcc -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- TORCH_BUILD_DIR: /home/rcvlab/anaconda3/lib/python3.5/site-packages/torch/lib
-- Found CUDA: /usr/local/cuda (found suitable version "8.0", minimum required is "6.5")
-- Autodetected CUDA architecture(s): 6.1 6.1
-- Torch_INSTALL_INCLUDE:/home/rcvlab/anaconda3/lib/python3.5/site-packages/torch/lib/include/home/rcvlab/anaconda3/lib/python3.5/site-packages/torch/lib/include/TH/home/rcvlab/anaconda3/lib/python3.5/site-packages/torch/lib/include/THC
-- Configuring done
-- Generating done
-- Build files have been written to: /media/rcvlab/soonmin2017/pytorch.segmentation/semantic/external/PyTorch-Encoding/encoding/lib
[100%] Building NVCC (Device) object CMakeFiles/ENCODING.dir/kernel/./ENCODING_generated_thc_encoding.cu.o
CMake Warning at /usr/share/cmake-2.8/Modules/FindCUDA/make2cmake.cmake:66 (message):
   Removing non-existent dependency file: generic/THStorage.h

CMake Warning at /usr/share/cmake-2.8/Modules/FindCUDA/make2cmake.cmake:66 (message):
   Removing non-existent dependency file: generic/THStorageCopy.h

CMake Warning at /usr/share/cmake-2.8/Modules/FindCUDA/make2cmake.cmake:66 (message):
   Removing non-existent dependency file: generic/THCStorage.h

CMake Warning at /usr/share/cmake-2.8/Modules/FindCUDA/make2cmake.cmake:66 (message):
   Removing non-existent dependency file: generic/THCStorageCopy.h

CMake Warning at /usr/share/cmake-2.8/Modules/FindCUDA/make2cmake.cmake:66 (message):
   Removing non-existent dependency file: generic/THTensor.h

CMake Warning at /usr/share/cmake-2.8/Modules/FindCUDA/make2cmake.cmake:66 (message):
   Removing non-existent dependency file: generic/THTensorCopy.h
...

/usr/lib/gcc/x86_64-linux-gnu/4.9/include/stddef.h(432): error: identifier "nullptr" is undefined
/usr/lib/gcc/x86_64-linux-gnu/4.9/include/stddef.h(432): error: expected a ";"
/usr/include/x86_64-linux-gnu/c++/4.9/bits/c++config.h(192): error: expected a ";"
/usr/include/c++/4.9/exception(63): error: expected a ";"

And finally,

/usr/include/c++/4.9/cmath(240): error: inline specifier allowed on function declarations only

/usr/include/c++/4.9/cmath(240): error: variable "std::constexpr" has already been defined

/usr/include/c++/4.9/cmath(240): error: expected a ";"

Error limit reached.
100 errors detected in the compilation of "/tmp/tmpxft_0000331f_00000000-7_thc_encoding.cpp1.ii".
Compilation terminated.
CMake Error at ENCODING_generated_thc_encoding.cu.o.cmake:264 (message):
  Error generating file
  /media/rcvlab/soonmin2017/pytorch.segmentation/semantic/external/PyTorch-Encoding/encoding/lib/CMakeFiles/ENCODING.dir/kernel/./ENCODING_generated_thc_encoding.cu.o


make[2]: *** [CMakeFiles/ENCODING.dir/kernel/./ENCODING_generated_thc_encoding.cu.o] Error 1
make[1]: *** [CMakeFiles/ENCODING.dir/all] Error 2
make: *** [all] Error 2
Traceback (most recent call last):
  File "setup.py", line 69, in <module>
    'install': install,
  File "/home/rcvlab/anaconda3/lib/python3.5/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/home/rcvlab/anaconda3/lib/python3.5/site-packages/setuptools/dist.py", line 338, in __init__
    _Distribution.__init__(self, attrs)
  File "/home/rcvlab/anaconda3/lib/python3.5/distutils/dist.py", line 281, in __init__
    self.finalize_options()
  File "/home/rcvlab/anaconda3/lib/python3.5/site-packages/setuptools/dist.py", line 471, in finalize_options
    ep.load()(self, ep.name, value)
  File "/home/rcvlab/anaconda3/lib/python3.5/site-packages/cffi/setuptools_ext.py", line 188, in cffi_modules
    add_cffi_module(dist, cffi_module)
  File "/home/rcvlab/anaconda3/lib/python3.5/site-packages/cffi/setuptools_ext.py", line 49, in add_cffi_module
    execfile(build_file_name, mod_vars)
  File "/home/rcvlab/anaconda3/lib/python3.5/site-packages/cffi/setuptools_ext.py", line 25, in execfile
    exec(code, glob, glob)
  File "/media/rcvlab/soonmin2017/pytorch.segmentation/semantic/external/PyTorch-Encoding/build.py", line 36, in <module>
    subprocess.check_call(build_all_cmd, env=dict(os.environ))
  File "/home/rcvlab/anaconda3/lib/python3.5/subprocess.py", line 581, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bash', 'encoding/make.sh']' returned non-zero exit status 2

I guess it's a gcc compiler issue, but I'm not sure.
Could you give me a hint for it?

Spelling mistake for DataLoaders

Hi, this is just a small mistake but can cause confusion when writing custom dataloaders.

main.py is looking for the class DataLoder instead of DataLoader

Thanks!

pytorch 0.3.1 supported?

(separated from here since touching a new topic)

When using this package with pytorch 0.3.1, compiled from source (not current master, but the source code corresponding to the official 0.3.1 release), it produces the following error message during training:

  [...]
    my_loss.backward()
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/autograd/variable.py", line 167, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph, retain_variables)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/autograd/__init__.py", line 99, in backward
    variables, grad_variables, retain_graph)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/autograd/function.py", line 91, in apply
    return self._forward_cls.backward(self, *args)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/encoding/functions/syncbn.py", line 47, in backward
    gradInput, input.data.view(B, C, -1), gradSum, gradSquare)
  File "/home/<user>/.conda/envs/pytorch-0.3.1-3.5-s/lib/python3.5/site-packages/torch/utils/ffi/__init__.py", line 180, in safe_call
    result = torch._C._safe_call(*args, **kwargs)
TypeError: initializer for ctype 'struct THCudaTensor *' must be a cdata pointer, not Variable

This can be fixed by changing syncbn.py, line 46-47 to: (addition of .data)

                encoding_lib.Encoding_Float_sum_square_Backward(
                    gradInput.data, input.data.view(B, C, -1), gradSum.data, gradSquare.data)

This is because in newer versions of pytorch (0.4), the Variable and Tensor class are merged into a single class, so there is no need to call .data.

From this I conclude that PyTorch-Encoding was written and is meant to be used with pytorch 0.4.

However, since pytorch 0.4 contains a lot of (even undocumented) breaking changes, some prefer to continue to use 0.3.1.

I kindly want to ask you if you think it is safe / supported to use your software with pytorch 0.3.1 (with the modification above that fixes the runtime error) or if you think there will be unexpected behaviour, degeneration in performance or similar. Because there is always a difference between "it runs without crashing" and "it runs as intended".

Support for HalfTensor

Hi, your work is really helpful to me, it's a incredible project. But when I found that Pytorch-Encoding don't support HalfTensor, it brought me some pain. My model applying synchronous batch normalization with FloatTensor got a satisfied outcome, so I want to enlarge batch size and slim the model size by using HalfTensor and compare this two models. Limited by my knowledge on cuda, I couldn't figure it out myself, so I come here telling my request. I know that you're quite busy, if this disturb you, I'm sorry.

zhanghang1989 / pytorch-encoding Goto Github PK

pytorch-encoding's Introduction

PyTorch-Encoding

Citations

pytorch-encoding's People

Contributors

Stargazers

Watchers

Forkers

pytorch-encoding's Issues

i try to use the DeepTen. where is module 'model.encodingnet' ?

Recommend Projects

Recommend Topics

Recommend Org

i try to use the DeepTen.
where is module 'model.encodingnet' ?