delta2323 / chainer-deepmark Goto Github PK

Currently we evaluate each architecture only with CPU or single GPU. As Soumith wrote, deepmark wants to support multiple GPU evaluation. The task is feasibility of multiple GPU evaluation with Chainer and implementation of it if possible.

There are (at least) two versions of AlexNet:

• ILSVRC2012 version (Paper, Caffe prototxt)
• One Weird Trick (OWT) version (Paper, cuda-convnet2)

Unfortunately, current implementation of AlexNet in is neither of them. We need to modify the architecture.

Note that in OWT version, the architecture in cuda-convnet2 is slightly different from paper version (footnote 1). Namely, channel num of conv4 is 256 vs. 384. We need to make clear which version to use.

Currently the architecture of BigLSTM is temporary one. We have to replace its architecture with the one in the paper

Rel #2

Current Chainer does not have Convolution3D and max_pooling_3d. C3D in chainer_deepmark now uses mocked ones.

hello,
this chainer-deepmark is a release-ready product? Do you(PFN) have any plan to officially publish it?
Thanks in advance.

Chainer supports Trainer feature from v1.11.0. But current Chainer-Deepmark writes training loops manually.

I am informed through personal communication that alex-owt can improve its throughput by 40% by increasing the workspace size. As the maximum number of batchsize is the one of the metrics of deepmark, the priority of throughput and memory usage depends on the situation. So it is better to prepare the choices that varies the size of workspace used in the evaluation scripts.

The result of measurement is required to output in JSON format. As the detail of the specification is not discussed enough, we will implement it after the specification will be determined.

Current implementation of VGG in chainer-deepmark is VGG-A in the original paper which is used in convnet-benchmarks. But we will use VGG-D in DeepMark.

Specification of DeepSpeech2 is found in DeepMark/deepmark#1.