@MarkWuNLP 谢谢
感觉训练数据不应该是一问一答或者多问一答的样子的么？

If I'm correct the I obtain a vocabulary size larger than 600.000 distinct words from the Ubuntu dataset. That number seems very big. Do you limit the vocabulary size for the model?

How did you tokenize the raw Douban corpus? I'm trying to test my model with new data, I need to tokenize them firstly.

I am having an issue with SMN_Dynamic

pygpu.gpuarray.GpuArrayException: ('mismatched shapes', 2)
Apply node that caused the error: GpuDot22(GpuReshape{2}.0, <GpuArrayType<None>(float32, matrix)>)
Toposort index: 756
Inputs types: [GpuArrayType<None>(float32, matrix), GpuArrayType<None>(float32, matrix)]
Inputs shapes: [(2000, 200), (100, 50)]
Inputs strides: [(40000, 4), (200, 4)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[GpuReshape{3}(GpuDot22.0, MakeVector{dtype='int64'}.0)]]

I'm calling train with 200 for the hidden size and embedding:
train(datasets,wordvecs.W,batch_size=200,max_l=max_word_per_utterence ,hidden_size=200,word_embedding_size=200,model_name='SMN_Dynamic.bin')
Can you help please ?

您好，全部的豆瓣训练语料能对外公开么，谢谢~

Hi, in your script PreProcess.py, if we use the DoubanConversaionCorpus as you released, where's the coressponding file word2vec.model when we call the class WordVecs?

We release Douban Conversation Corpus, comprising a training data set, a development set and a test set

I can only see the test set (test.txt) and a small sample of the training set. Where is the rest of the training (and validation) set?

为什么添加矩阵A，而不是像M1一样直接相乘呢？

hi 您好

如何把豆瓣开源的对话数据应用到这个multiTurnResponseSelection上面呢？
十分感谢

Thanks
weizhen

https://github.com/MarkWuNLP/MultiTurnResponseSelection/blob/master/src/SMN_Dynamic.py#L95

多谢多谢@MarkWuNLP

您好！感谢您对数据集的公开！关于这份数据集，请问是否有未去除标点符号的版本？

Hi Wu,

We tried to carefully replicate your work in tensorflow and the performances are awesome, however, our model is still 3-4% lower compared to you reported in your paper.

We suspect the reason is that the word embedding for initialization we used is different to yours. Thus I wonder if you could provide the train/test binary file that you used in your code?

Best wishes,
Xijuan

I want to train the model on another dataset, how should I process the data to generate the .pkl files?

how is the training file you shared supposed to be read? shall I parse it myself? or have you used a library to store it? I'm talking about this file: https://1drv.ms/u/s!AtcxwlQuQjw1jF0bjeaKHEUNwitA

在tensorflow版本中，actions从来没有进入到模型训练这一环节。总是把true_utt给response的placeholder。

你们的工作对我启发很大，在github上只看到了豆瓣数据的train sample，请问一下完整的数据会发布在哪里，何时发布哦？

dataset = r"../ubuntu_data.mul.100d.fullw2v.train"

response candidate r 我理解应该是固定的一句话，现在是把r这整句话一起输进SMN，

如果换成r的id会怎样？

@MarkWuNLP 谢谢！

onedrive上的URL 失效了，不知道能不能解决一下。感谢

你好，数据集onedrive下载链接失效了，请问可以回复一下吗？

谢谢！
@MarkWuNLP

您好，按照您在readme里面写的，修改path后运行PreProcess.py提示错误，找不到word2vec.model，请问在PreProcess.py中用到的word2vec.model，以及其他文件：train.topic、mergedic2.txt等文件在哪里呢？谢谢！

这里的
https://github.com/MarkWuNLP/MultiTurnResponseSelection/blob/master/src/PreProcess.py#L149
谢谢谢谢@MarkWuNLP

Thank you for sharing the code for your paper. I was wondering how many epochs you trained the model on Ubuntu dialogue corpus for? Was training done on a single GPU? If so, how long would a single epoch take on average?

@MarkWuNLP 谢谢谢谢

您所了解的情况
多谢
多谢
多谢
@MarkWuNLP

可以麻烦您重新传一份吗

Hi, Yu,

I followed the instructions in #14 to run SMN_Last.py. But it does not work. As you suggested, I modified the parameter word_embedding_size=200 hidden_size=200 in SMN_Last.py.

The following is the error message. Do you know the reason for this? Thanks!

trainning data 949999 val data 50001
image shape (200, 2, 50, 50)
filter shape (8, 2, 3, 3)
/mnt/scratch/lyang/working/PycharmProjects/NLPIRNNMatchZooQA/src-match-zoo-lyang-dev/matchzoo/conqa/smn_yuwu_acl17/src/CNN.py:233: UserWarning: DEPRECATION: the 'ds' parameter is not going to exist anymore as it is going to be replaced by the parameter 'ws'.
self.output =theano.tensor.signal.pool.pool_2d(input=conv_out_tanh, ds=self.poolsize, ignore_border=True,mode="max")
(200, 50)
[[ -7.18495249e-03 1.15389853e-02 -4.65663650e-03 ..., 1.29073645e-02
5.36817317e-03 -3.73283600e-03]
[ -7.56547710e-04 2.52639169e-03 2.14198747e-03 ..., 1.87613393e-03
3.13413644e-03 -1.80455989e-03]
[ 1.70666858e-02 -1.36987258e-02 2.04090084e-02 ..., 6.40176979e-03
2.26856302e-02 -3.21805640e-02]
...,
[ 1.94880138e-02 1.08157883e-03 7.03368021e-05 ..., 1.80350402e-02
-3.28865572e-03 -1.50815454e-02]
[ 5.96401096e-03 -9.91006641e-04 4.77976783e-03 ..., 2.32200795e-02
3.23144545e-02 -8.65365822e-03]
[ -6.93387402e-04 3.27511476e-03 2.07571058e-03 ..., 1.42423584e-03
3.12527304e-03 8.35354059e-04]]
Traceback (most recent call last):
File "SMN_Last.py", line 422, in
,hidden_size=200,word_embedding_size=200)
File "SMN_Last.py", line 334, in train
train_model = theano.function([index], cost,updates=grad_updates, givens=dic,on_unused_input='ignore')
File "/usr/lib/python2.7/site-packages/theano/compile/function.py", line 326, in function
output_keys=output_keys)
File "/usr/lib/python2.7/site-packages/theano/compile/pfunc.py", line 449, in pfunc
no_default_updates=no_default_updates)
File "/usr/lib/python2.7/site-packages/theano/compile/pfunc.py", line 208, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<TensorType(float32, matrix)>, shared_var.type=TensorType(float32, matrix), update_val=Elemwise{add,no_inplace}.0, update_val.type=TensorType(float64, matrix)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

您好，阅读您的文献的时候发现了您开源的多轮对话语聊，请问能不能把训练集和验证集的语聊发我一下？([email protected]) 非常感谢！

Hi， 我clone下来的代码跑挂了，没弄懂什么原因：

MultiTurnResponseSelection/src/CNN.py:233: UserWarning: DEPRECATION: the 'ds' parameter is not going to exist anymore as it is going to be replaced by the parameter 'ws'.
self.output =theano.tensor.signal.pool.pool_2d(input=conv_out_tanh, ds=self.poolsize, ignore_border=True,mode="max")
Traceback (most recent call last):
File "SMN_Last.py", line 422, in
,hidden_size=100,word_embedding_size=100)
File "SMN_Last.py", line 334, in train
train_model = theano.function([index], cost,updates=grad_updates, givens=dic,on_unused_input='ignore')
File "/usr/lib64/python2.7/site-packages/theano/compile/function.py", line 326, in function
output_keys=output_keys)
File "/usr/lib64/python2.7/site-packages/theano/compile/pfunc.py", line 449, in pfunc
no_default_updates=no_default_updates)
File "/usr/lib64/python2.7/site-packages/theano/compile/pfunc.py", line 208, in rebuild_collect_shared
raise TypeError(err_msg, err_sug)
TypeError: ('An update must have the same type as the original shared variable (shared_var=<TensorType(float32, matrix)>, shared_var.type=TensorType(float32, matrix), update_val=Elemwise{add,no_inplace}.0, update_val.type=TensorType(float64, matrix)).', 'If the difference is related to the broadcast pattern, you can call the tensor.unbroadcast(var, axis_to_unbroadcast[, ...]) function to remove broadcastable dimensions.')

这个是theano的版本问题么？

it's a good job, you meant that you have released the douban corpus in the paper, but i couldn't find the training data in this project.

希望README文件能够详细一点，尤其是对于使用到的数据的描述。

如何我没有理解错的话，Ubuntu Dialogue Corpus和豆瓣数据是 一个帖子题目 加 该帖子的N条回复，
为什么前面可以当作context 以及 最后一条回复可以当作response？
十分谢谢谢谢@MarkWuNLP

我训练了2个epoch，loss=0.334, R10_1=0.646, R2_1=0.895，而且奇怪的是后两个指标在训练过程中一直不变。大家训练的loss能下降到多少呢？performance可以达到论文中的数值吗？
我用的tensorflow版本的。