I have got an error when I used raw_shape instead of shape as below code

shape = (384, 160)
img = cv2.imread('demo.jpg')
raw_shape = (img.shape[1], img.shape[0])

img = cv2.resize(img, shape)

img = cv2.resize(img, raw_shape)

Thank you piiswrong

Hi,

I'm getting this error: Find name bn_pool3_moving_inv_var that is not in the auxiliary states

Would you know how to resolve it?

Thanks!

mxnet 2.0.0

Hi,

anyone can tell me the right, TESTED and WORKING configuration under Ubuntu?

This config is work???
Ubuntu 16.04LTS -64bit
Cuda 8
DNN 5.1
Nvidia 1080-8gb
mxnet from git
deep3d from git

Best
Moty

Dear,

I am using visual studio 2015 with python tools, all the imports in the code are running well except mxnet and leveldb

I think I have an issue in setting up mxnet I followed the official document and still there is an error can you help please

Regards,

The latest mxnet seems not have the DepthDot operator

@piiswrong
I use the parse.py and data.py to create my own database. I just give one prefix. But when I run the data.py， it needs to get the prefix_list,. How can I solve it ? Thanks.

I ran the convert_movie.py file and it generateed this error. It seems that it is because of the version of opencv I used. Can you please check which opencv version are you using? I am using v2.4.8. Thanks a lot!

Hello,I can't find these:
http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params &&
eep3d-0050.params
I don't know what's the matter.
can you send me your params? Tank you !
mail to : [email protected]

Hi,

I'm trying to run convert_movie.py using a single jpg as an input file and the default params file, however, I get the above error. Has anyone had this problem before?

thanks
laszlo

edit:
I set my args such as (all my paths and files are correct):

python convert_movie.py C:\dev\mxnet\deep3d\param --ctx 0 --source C:\dev\mxnet\deep3d\image\someImage.jpg --output C:\dev\mxnet\deep3d\output\out.jpg

I got this error when add depth-dot code and recomplie mxnet, and change 'type' to 'mode' in softmax

what is the mean? i hope to get you help
"Then, append EXTRA_OPERATORS=path/to/deep3d/operators to path/to/mxnet/config.mk and recompile MXNet."

I'd like to train the network using KITTI stereo datasets instead of stereo movies. is that somehow possible - and if yes, can someone give me some pointers? thanks, Ito

[12:38:08] /root/mxnet/dmlc-core/include/dmlc/logging.h:245: [12:38:08] src/operator/operator.cc:19: Cannot find Operator CuDNNBatchNorm in registry
Traceback (most recent call last):
File "myMain.py", line 23, in
model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.py", line 822, in load
symbol, arg_params, aux_params = load_checkpoint(prefix, epoch)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.py", line 362, in load_checkpoint
symbol = sym.load('%s-symbol.json' % prefix)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/symbol.py", line 886, in load
check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.5.0-py2.7.egg/mxnet/base.py", line 77, in check_call
raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: Failed loading Op bn_pool1 of type CuDNNBatchNorm: [12:38:08] src/operator/operator.cc:19: Cannot find Operator CuDNNBatchNorm in registry

cuda7.5 cudnn4
running by official mxnet docker gpu image

Thank you piiswrong

when I run the' Y = model.predict(test_iter)' ,which has NameError: name 'model' is not defined. how to solve it ?

Hi,
are there any plans to update the code with the newest version of MxNet and with Cuda 5?

Cheers!

MXNetError: [16:38:44] src/operator/cudnn_batch_norm.cc:20: CuDNNBatchNorm is merged into BatchNorm for cudnn version above v5.Use the later instead.

any working solution? or patch?

Although I mansged to run through the test code with my own dataset, the result is very poor, the right eye images are rather blur compared with left one. Could you please share the file mean.npz ? My test data set is very small and may be the mean values generated from it are not very good?

hello, i had met a new problem about the data format error, like this.

would love to try generate the parameters by training module, are you releasing it?

thanks!

i build the code in window7 with vs2013, and got .dll and .lib , but i con not know how to run it to see the result ,can you give me a favor?

I run the code according to the deep3d.ipynb, but i found the result was not as good as your sample which you provided (that is the demo.gif).
can you do me a favor and tell me what can i do to get a result as good as you provided.

thank you!

I had create a database using parse.py and data.py, but when I run train.py I got this error:

$ python train.py [01:08:37] include/dmlc/logging.h:235: [01:08:37] src/io/local_filesys.cc:149: Check failed: allow_null LocalFileSystem: fail to open "vgg16-0001.params" Traceback (most recent call last): File "train.py", line 92, in <module> train(64, 'exp/deep3d') File "train.py", line 65, in train vgg16 = data.load_vgg(data_frames, flow_frames, two_stream=False) File "/home/salim/deep3d/data.py", line 443, in load_vgg vgg16 = {name: arr for name, arr in mx.nd.load('vgg16-0001.params').items() if name.startswith('arg:conv')} File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/ndarray.py", line 956, in load ctypes.byref(names))) File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/base.py", line 77, in check_call raise MXNetError(py_str(_LIB.MXGetLastError())) mxnet.base.MXNetError: [01:08:37] src/io/local_filesys.cc:149: Check failed: allow_null LocalFileSystem: fail to open "vgg16-0001.params"

What file is that vgg16-0001.params???? What have to do make it work?
Thanks.

which version of Mxnet are you supposed to use ?

Can you give me some codes about that? My ability is so poor that I can't find the way to output these maps. I try to output these maps from the deconvolution which is behind some pooling layer, but I failed, I can't find the way to output some values such as pred1, pred2, and so on in sym.py from the module. I just want the method or codes of the output of the 4-by-3 maps from the depth layers in your paper. Can you help me? Thank you.

Hi ，@piiswrong
[15:44:47] /home/git/mxnet/mxnet/dmlc-core/include/dmlc/logging.h:235: [15:44:47] src/operator/./cudnn_softmax_activation-inl.h:44: Check failed: (in_data[softmax_activation::kData].ndim()) == (2) Input need to have 2 dimensions when mode=instance.

[15:44:47] /home/git/mxnet/mxnet/dmlc-core/include/dmlc/logging.h:235: [15:44:47] src/engine/./threaded_engine.h:306: [15:44:47] src/operator/./cudnn_softmax_activation-inl.h:44: Check failed: (in_data[softmax_activation::kData].ndim()) == (2) Input need to have 2 dimensions when mode=instance.

cuda7.5+cudnn5.0
I get this result when I run "python ./train.py".
How to solve this problem, thank you

I was wondering how much VRAM this network requires for the batchsize 64? The reason I ask is that I am working on porting this to tensorflow for a class project, and am currently restricted to a batch size of 32 due to memory issues on a K80 with ~11.2 gb available VRAM.

Also, were all the operations performed on GPU in this mxnet implementation? Any information you could provide on memory usage would be much appreciated!

Hi,

Can someone tell ma what the structure is creating databases for deep3d, and how to use this to train a model.

thanks,
laszlo

excuse me, which option need to be configured or set for this error?

Hi @piiswrong
Looks the pre-trained model is difficult to run with newer version of cuda and cudnn. Maybe I would try to train a new deep3d model with cuda 8.0 and cudnn 5.1. Can you please give me some suggestions about this task?

Thanks very much!

I would like to know if it is still available these network parameters?
Best,
Goffredo

if not os.path.exists('deep3d-0050.params'):
    urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

The error gives:

---------------------------------------------------------------------------
MXNetError                                Traceback (most recent call last)
<ipython-input-7-9deaeed89c01> in <module>()
      1 if not os.path.exists('deep3d-0050.params'):
      2     urllib.urlretrieve('http://homes.cs.washington.edu/~jxie/download/deep3d-0050.params', 'deep3d-0050.params')
----> 3 model = mx.model.FeedForward.load('deep3d', 50, mx.gpu(0))

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load(prefix, epoch, ctx, **kwargs)
    832         - ``prefix-epoch.params`` will be saved for parameters.
    833         """
--> 834         symbol, arg_params, aux_params = load_checkpoint(prefix, epoch)
    835         return FeedForward(symbol, ctx=ctx,
    836                            arg_params=arg_params, aux_params=aux_params,

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/model.pyc in load_checkpoint(prefix, epoch)
    361     - parameters will be loaded from ``prefix-epoch.params``.
    362     """
--> 363     symbol = sym.load('%s-symbol.json' % prefix)
    364     save_dict = nd.load('%s-%04d.params' % (prefix, epoch))
    365     arg_params = {}

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/symbol.pyc in load(fname)
    932         raise TypeError('fname need to be string')
    933     handle = SymbolHandle()
--> 934     check_call(_LIB.MXSymbolCreateFromFile(c_str(fname), ctypes.byref(handle)))
    935     return Symbol(handle)
    936 

/home/jjs/anaconda3/envs/tf/lib/python2.7/site-packages/mxnet-0.5.0-py2.7.egg/mxnet/base.pyc in check_call(ret)
     75     """
     76     if ret != 0:
---> 77         raise MXNetError(py_str(_LIB.MXGetLastError()))
     78 
     79 def c_str(string):

MXNetError: Failed loading Op softmax of type SoftmaxActivation: Cannot find argument 'type', Possible Arguments:
----------------
mode : {'channel', 'instance'},optional, default='instance'
    Softmax Mode. If set to instance, this operator will compute a softmax for each instance in the batch; this is the default mode. If set to channel, this operator will compute a num_channel-class softmax at each position of each instance; this can be used for fully convolutional network, image segmentation, etc.```

It looks like:
```sym = symbol.load('deep3d-symbol.json')

How to fix it?
Thanks

I notice there is a variable named 'source' in convert movie.py , what is its meaning actually? What is the test_idx? And if i want to use the code to directly input a video's name, what extra work should be done to produce a 3D movie rather than a simple gif?

When I run the demo, I encounter the error: /mxnet/dmlc-core/include/dmlc/logging.h:245: [18:04:24] src/operator/operator.cc:19: Cannot find Operator CuDNNBatchNorm in registry.

Thanks a lot!

From your README.md:

Following each image, there are 4-by-3 maps of depth layers, ordered from near
to far. You can see that objects that are near to you appear in the first depth
maps and objects that are far away appear in the last ones. This shows that the
internal depth representation is learning to infer depth from 2D images without
been directly trained on it.

Where in the code do you generate depth layers from an image?

Can this algorithm be used to render a 2D video stream into 3D in real time?

i run the demo successfully， but i can't open the demo.gif that the python code gernerate , the ie screen is as follows:

I'd like to train the network using depth maps as ground truths. is that somehow possible - and if yes, can someone give me some pointers? thanks, laszlo

Hi,
I can't find the dataset (3d movies) mentioned in the paper. Can you illustrate how to download the dataset?

When I run the code to the place(in data.py next member function for class Mov3dStack):
for j in range(max(1, self.data_frames)):
sl = txn.get('%09d'%(idx+(j-self.data_frames/2)_self.stride), db=self.ldb)
if sl is None:
pass
else:
_, s = mx.recordio.unpack(sl)
mx.nd.imdecode(s, clip_rect=(p[0], p[1], p[0] + self.data_shape[0], p[1] +self.data_shape[1]),
out=ndleft, index=i_self.data_frames+j, channels=3, mean=self.left_mean_nd)

Then error thrown out as :
File "/media/lqzhu/e/deep3d-master/data.py", line 215, in load_mean
for batch in data_iter:
File "/media/lqzhu/e/deep3d-master/data.py", line 392, in next
out=ndleft, index=i*self.data_frames+j, channels=3, mean=self.left_mean_nd)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/ndarray.py", line 1034, in imdecode
out=out)
File "/usr/local/lib/python2.7/dist-packages/mxnet-0.7.0-py2.7.egg/mxnet/ndarray.py", line 1172, in generic_ndarray_function
c_array(ctypes.c_char_p, [str(i).encode('ascii') for i in kwargs.values()])))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xff in position 0: ordinal not in range(128)

I dig in the web and no useful dips can fix this problem, I've tried adding below lines:
import sys
reload(sys)
sys.setdefaultencoding('utf8')

Still doesn't work.
Any suggestion?

Can you provide a more detailed instruction on how to run your code?

Please, include a License.

Hello,
I'm currently implementing your great in TF using the official params but the output is so bad and i don't know why ?
Could you please give me any advice to make the output better?
input>

output>

we cann't connect the url https://www.cs.washington.edu/~jxie/.

pls help me.

I wanna ask that how to initialize the weight of deconvolution filter to bilinear interpolati as you did In your newest paper "Deep3D: Automatic 2D-to-3D Video Conversion with CNNs"?
Thanks a lot!