zhiguowang / bimpm Goto Github PK

1       1:0.38222232461 0:0.617777705193

0       1:0.669489085674 0:0.330510884523

0       1:0.894043326378 0:0.105956666172

1       1:0.770928144455 0:0.229071870446

0       1:0.691392481327 0:0.30860760808

My guess is that, the first column is the predicted label, the second column is label 1 with its probability, and third column is label 0 with its probability. However, if it's this way, then the first column should contain the label whichever has a larger probability in the second and the third column. It is not. The last line 0 1:0.691392481327 0:0.30860760808 contradicts my guess.

So how should I interpret this?

I got this error when run:
python SentenceMatchTrainer.py --config_path /home/tamiomusic/BiMPM/con
figs/quora.sample.config

I only change quora.sample.config to fix with files' directory:
"train_path": "/home/tamiomusic/BiMPM/quora/train.tsv",
"dev_path": "/home/tamiomusic/BiMPM/quora/dev.tsv",
"word_vec_path": "/home/tamiomusic/BiMPM/quora/wordvec.txt",
"model_dir": "/home/tamiomusic/BiMPM/quora/logs"

Full log:
Loading the configuration from /home/tamiomusic/BiMPM/configs/snli.sample.config
Traceback (most recent call last):
File "SentenceMatchTrainer.py", line 251, in
FLAGS = enrich_options(FLAGS)
File "SentenceMatchTrainer.py", line 201, in enrich_options
if not options.dict.has_key("in_format"):
AttributeError: 'dict' object has no attribute 'has_key'

Best regards,

I want to know if you have done experiments on MSRP dataset, if yes, can you share the configuration files. Thanks very much!

hi zhiguowang !
why not use prob after softmax?

Should line 62 of SentenceMatchDataStream.py be:

if len(word_idx_2)>max_sent_length:
    word_idx_2 = word_idx_2[:max_sent_length]
    char_matrix_idx_2 = char_matrix_idx_2[:max_sent_length] # char_matrix_idx_2, instead of char_matrix_idx_1

Train in epoch 0
2018-08-09 13:45:18.434266: F ./tensorflow/core/util/cuda_kernel_helper.h:160] Check failed: work_element_count > 0 (0 vs. 0)

When I use the similar config to quora.sample.config, it can works.
But when I change "fix_word_vec" to false for retraining wordvec, failed occurs.

Cannot create a tensor proto whose content is larger than 2GB.
The problem occured where word token matrix feed into a tensor:
File "/SentenceMatchModelGraph.py", line 35,

I noticed that you are updating the word embedding while training.
Then why you are not saving it . Because of not saving it, when I restore the model, it gets randomly initialized and thus gives random results.

When I trained the model with GPU and the training data is very bigger, I found low utilization rate of GPU, namely about 11% and the utilization rate of CPU is about 110%. I want to know How to increase the utilization rate of GPU? The batch_size cannot be bigger because of the limited memory.

I also want to know What are you like when you train because RNN layers used in the model would slow training speed.

Number of instances in devDataStream: 16855
Number of batches in devDataStream: 57
WARNING:tensorflow:From ///BiMPM_tf1.5/src/match_utils.py:150: calling reduce_max (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From ///BiMPM_tf1.5/src/match_utils.py:151: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead
WARNING:tensorflow:From ///BiMPM_tf1.5/src/SentenceMatchModelGraph.py:156: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.
Instructions for updating:

Future major versions of TensorFlow will allow gradients to flow
into the labels input on backprop by default.

See tf.nn.softmax_cross_entropy_with_logits_v2.

Traceback (most recent call last):
File "src/SentenceMatchTrainer.py", line 296, in
main(FLAGS)
File "src/SentenceMatchTrainer.py", line 217, in main
is_training=True, options=FLAGS, global_step=global_step)
File "///BiMPM_tf1.5/src/SentenceMatchModelGraph.py", line 10, in init
self.create_model_graph(num_classes, word_vocab, char_vocab, is_training, global_step=global_step)
File "///BiMPM_tf1.5/src/SentenceMatchModelGraph.py", line 166, in create_model_graph
l2_loss = tf.add_n([tf.nn.l2_loss(v) for v in tvars if v.get_shape().ndims > 1])
File "//BiMPM_tf1.5/src/SentenceMatchModelGraph.py", line 166, in
l2_loss = tf.add_n([tf.nn.l2_loss(v) for v in tvars if v.get_shape().ndims > 1])
TypeError: '>' not supported between instances of 'NoneType' and 'int'

I implement a simplified keras version here. However the val_loss is not very good compared to a simple lstm network without matching layer. Could you help me to check it out if my implementation has any bugs? Thanks.

I see layer_utils.py has a func named cross_entropy,did you use it for calculate the loss of wikiqa?
Could you please also share the configuration file for the WikiQA and TrecQA experiments to achieve your best results in the paper? Thank you very much!

Is it possible to re-train a model? If so how?

遇到一个问题, 在full match layer的输出中 , 论文中:
sentence1_match_output shape = batchsize * len(sentence2) * L (num perspective)
sentence2_match_output shape = batchsize * len(sentence1) * L (num perspective)
然后作为 aggregation 中的 lstm 的输入, 但是代码中了 几个其他的操作, 比如代码中默认 L = 5, 最后得到的 维度是 batchsize * len(sentence) * 18, 请问还加了什么操作? 论文里的结果是加了这些其他的操作的结果吗?

谢谢

Hi , zhiguowang!
I found that you use dropout in training and when testing with out dropout.

When the model is trained, add dropout. If the test is not added, the probability of the test will be more than 1.

(venv) mldl@mldlUB1604:~/ub16_prj/BiMPM$ python src/SentenceMatchTrainer.py --config_path configs/snli.sample.config
Loading the configuration from configs/snli.sample.config
Collecting words, chars and labels ...
Number of words: 35012
Number of chars: 61
word_vocab shape is (35611, 300)
Number of labels: 3
Build SentenceMatchDataStream ...
Number of instances in trainDataStream: 549367
Number of batches in trainDataStream: 5494
Number of instances in devDataStream: 9842
Number of batches in devDataStream: 99
Traceback (most recent call last):
File "src/SentenceMatchTrainer.py", line 253, in
main(FLAGS)
File "src/SentenceMatchTrainer.py", line 175, in main
is_training=True, options=FLAGS, global_step=global_step)
File "/home/mldl/ub16_prj/BiMPM/src/SentenceMatchModelGraph.py", line 10, in init
self.create_model_graph(num_classes, word_vocab, char_vocab, is_training, global_step=global_step)
File "/home/mldl/ub16_prj/BiMPM/src/SentenceMatchModelGraph.py", line 97, in create_model_graph
is_training=is_training, dropout_rate=options.dropout_rate, use_cudnn=options.use_cudnn)
File "/home/mldl/ub16_prj/BiMPM/src/layer_utils.py", line 20, in my_lstm_layer
name="{}_cudnn_bi_lstm".format(scope_name), dropout=dropout_rate if is_training else 0)
TypeError: init() got an unexpected keyword argument 'name'

Training works, but it seems very slow... which is fine, as long as this is the expected behavior. I'm just curious if it is unusually slow for me.... did you happen to get this warning?

/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
  "Converting sparse IndexedSlices to a dense Tensor of unknown shape. "

If not, then I'm wondering if my code is running slow due to this message...
So far, I know it's coming from the tf.gradients() function call in line 220 of SentenceMatchModelGraph.py:

grads, _ = tf.clip_by_global_norm(tf.gradients(self.loss, tvars), clipper)

Unfortunately, this doesn't help much because I still don't know which part of the network is triggering it...
If you tell me that your are not getting this message, then I will investigate further and try to find the root cause. Thanks!

hello, When I run the command python BiMPM/src/SentenceMatchTrainer.py, it shows error:

Traceback (most recent call last):
  File "BiMPM/src/SentenceMatchTrainer.py", line 12, in <module>
    from SentenceMatchModelGraph import SentenceMatchModelGraph
  File "/home/xxx/BiMPM/src/SentenceMatchModelGraph.py", line 2, in <module>
    import my_rnn
  File "/home/xxx/BiMPM/src/my_rnn.py", line 12, in <module>
    from tensorflow.python.ops import rnn_cell
ImportError: cannot import name rnn_cell

can you update your code for tensorflow1.0.
thank you very much!!

Now that I have the model trained and ready, how can I let it predict the labels of unlabelled questions?

In your instructions, you say that the test and trainings sets need to be in the following format:

label sentence#1 sentence#2 other_info

But if I want to run the model on unlabelled data, then I won't know the value of "label". How can I build my file and have the trained model predict the labels? I would have put dummy values for "label" but I'm worried it might affect the model's performance later on. Or is that not the case?

In the framework， before the BiLSTM layer（context representive and aggragation layer）， there are a highway network.
So, I want to know the reason that add this highwat layer?

Hi,
I'm trying to reproduce your result on SNLI - can we have the configuration files for that? Some configurations seems unclear to me. E.g., NER_dim, POS_dim, max_char_per_word, etc. Can we reproduce all the results using the default parameters? Thank you very much!

Can someone please tell me what the Word Vector (i.e. the input for the --word_vec_path flag) file should contain? I read through the paper but I didn't find any useful indications.

I trained my model and got a 82% accuracy on dev set. Then I tried to evaluated it on test set, but it gave me 37% accuracy. I thought it was very strange, so I also tried to run SentenceMatchDecoder.py on my dev set, it again gave a 36% accuracy, which is not reasonable because it should be 82%. Then I found the correct_tags in SentenceMatchTrainer.evaluate() actually claculated the number of pairs which originally have label '1' but not the correct # of predict. I tried to go through your code in SentenceMatchGraph.py but still have no idea of what's wrong. Can you help me to figure it out?

What are the factors that affects the evaluation speed of the model significantly without improving the performance much? in other words, what parameter to adjust to make the model evaluates faster without sacrificing too much accuracy? figuring this out by trials and errors is too time consuming, and any comment or suggestion is welcomed. Thank you!

Hi Dr. Wang,
I notice that paddings for sentences in each batch are zeros (idx)

but the pre-trained embeddings are declared with the last column to be all zeros (which I assume to be used for the word?), and all proceeding ones read from pre-trained embedding file.

So during the embedding lookup step, paddings are actually using the embedding of the first word stored in the pre-trained embedding file, which I think should be the last instead.

The subroutine can be traced here:
(1) word_vocab is constructed with 'txt3'

Please correct me if I'm wrong and hope to hear from you!
Thank you!

Is it possible to publish other models in your paper? I want to calculate F1-score and Significant test and need this models :(

I tried to start training the model by using the default configuration file for quora. This has use_cudnn=true. But it has run into some unexpected error, when I run the SentenceMatchTrainer.py file. The error is as follows:

(tensorflowGPU) D:\Back Up\Desktop\Setiment Analysis\synonym_paraphrase\BiMPM\src>python SentenceMatchTrainer.py --config_path "../configs/quora.sample.config"
Loading the configuration from ../configs/quora.sample.config

{'train_path': '../data/quora/train.tsv', 'dev_path': '../data/quora/dev.tsv', 
'word_vec_path': '../data/quora/wordvec.txt', 'model_dir': 'quora_model', 'suffix': 'quora', 'fix_word_vec': True, 'isLower': True, 'max_sent_length': 50, 'max_char_per_word': 10, 
'with_char': True, 'char_emb_dim': 20, 'char_lstm_dim': 40, 'batch_size': 60, 'max_epochs': 20, 'dropout_rate': 0.1, 'learning_rate': 0.0005, 'optimize_type': 'adam', 'lambda_l2': 0.0,
 'grad_clipper': 10.0, 'context_layer_num': 1, 'context_lstm_dim': 100,
 'aggregation_layer_num': 1, 'aggregation_lstm_dim': 100, 'with_full_match': True, 'with_maxpool_match': False, 'with_max_attentive_match': False, 'with_attentive_match': True, 
'with_cosine': True, 'with_mp_cosine': True, 'cosine_MP_dim': 5, 'att_dim': 50, 'att_type': 'symmetric', 'highway_layer_num': 1, 
'with_highway': True, 'with_match_highway': True, 
'with_aggregation_highway': True, 'use_cudnn': True, 'with_moving_average': False}

Collecting words, chars and labels ...
Number of words: 104891
Number of chars: 1198
word_vocab shape is (106686, 300)
Number of labels: 2
Build SentenceMatchDataStream ...
Number of instances in trainDataStream: 384348
Number of batches in trainDataStream: 6406
Number of instances in devDataStream: 10000
Number of batches in devDataStream: 167
2019-05-30 00:41:22.120164: I T:\src\github\tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-05-30 00:41:23.282409: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1356] Found device 0 with properties:
name: GeForce GTX 1060 with Max-Q Design major: 6 minor: 1 memoryClockRate(GHz): 1.48
pciBusID: 0000:01:00.0
totalMemory: 6.00GiB freeMemory: 4.97GiB
2019-05-30 00:41:23.289931: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1435] Adding visible gpu devices: 0
2019-05-30 00:41:25.325066: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:923] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-05-30 00:41:25.329970: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:929]      0
2019-05-30 00:41:25.332505: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:942] 0:   N
2019-05-30 00:41:25.337204: I T:\src\github\tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1053] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 4740 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1060 with Max-Q Design, pci bus id: 0000:01:00.0, compute capability: 6.1)
Traceback (most recent call last):
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 1322, in _do_call
    return fn(*args)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 1305, in _run_fn
    self._extend_graph()
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _extend_graph
    tf_session.ExtendSession(self._session)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot colocate nodes 'Model/global_norm/L2Loss_38' and 'Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop' because no device type supports both of those nodes and the other nodes colocated with them.
Colocation Debug Info:
Colocation group had the following types and devices:
CudnnRNNBackprop: GPU
L2Loss:

Colocation members and user-requested devices:
  Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop (CudnnRNNBackprop)
  Model/global_norm/L2Loss_38 (L2Loss)

         [[Node: Model/global_norm/L2Loss_38 = L2Loss[T=DT_FLOAT, _class=["loc:@Model...NNBackprop"]](Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop:3)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "SentenceMatchTrainer.py", line 257, in <module>
    main(FLAGS)
  File "SentenceMatchTrainer.py", line 191, in main
    sess.run(initializer)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 900, in run
    run_metadata_ptr)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 1135, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 1316, in _do_run
    run_metadata)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\client\session.py", line 1335, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot colocate nodes 'Model/global_norm/L2Loss_38' and 'Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop' because no device type supports both of those nodes and the other nodes colocated with them.
Colocation Debug Info:
Colocation group had the following types and devices:
CudnnRNNBackprop: GPU
L2Loss:

Colocation members and user-requested devices:
  Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop (CudnnRNNBackprop)
  Model/global_norm/L2Loss_38 (L2Loss)

         [[Node: Model/global_norm/L2Loss_38 = L2Loss[T=DT_FLOAT, _class=["loc:@Model...NNBackprop"]](Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop:3)]]

Caused by op 'Model/global_norm/L2Loss_38', defined at:
  File "SentenceMatchTrainer.py", line 257, in <module>
    main(FLAGS)
  File "SentenceMatchTrainer.py", line 175, in main
    is_training=True, options=FLAGS, global_step=global_step)
  File "D:\Back Up\Desktop\Setiment Analysis\synonym_paraphrase\BiMPM\src\SentenceMatchModelGraph.py", line 10, in __init__
    self.create_model_graph(num_classes, word_vocab, char_vocab, is_training, global_step=global_step)
  File "D:\Back Up\Desktop\Setiment Analysis\synonym_paraphrase\BiMPM\src\SentenceMatchModelGraph.py", line 175, in create_model_graph
    grads, _ = tf.clip_by_global_norm(grads, self.options.grad_clipper)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\ops\clip_ops.py", line 240, in clip_by_global_norm
    use_norm = global_norm(t_list, name)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\ops\clip_ops.py", line 179, in global_norm
    half_squared_norms.append(gen_nn_ops.l2_loss(v))
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\ops\gen_nn_ops.py", line 4679, in l2_loss
    "L2Loss", t=t, name=name)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\framework\ops.py", line 3392, in create_op
    op_def=op_def)
  File "C:\Users\derp\AppData\Local\conda\conda\envs\tensorflowGPU\lib\site-packages\tensorflow\python\framework\ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Cannot colocate nodes 'Model/global_norm/L2Loss_38' and 'Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop' because no device type supports both of those nodes and the other nodes colocated with them.
Colocation Debug Info:
Colocation group had the following types and devices:
CudnnRNNBackprop: GPU
L2Loss:

Colocation members and user-requested devices:
  Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop (CudnnRNNBackprop)
  Model/global_norm/L2Loss_38 (L2Loss)

         [[Node: Model/global_norm/L2Loss_38 = L2Loss[T=DT_FLOAT, _class=["loc:@Model...NNBackprop"]](Model/gradients/Model/aggregation_layer/right_layer-0/right_layer-0_cudnn_bi_lstm/CudnnRNN_grad/CudnnRNNBackprop:3)]]

When I set use_cudnn:false, the training starts without any problems. In this case, it is still using the GPU. I understand from the code that use_cudnn=true helps make use of the CudnnLSTM, but maybe the issue arises due to OS or the Tensorflow version. The details of the environment are:
OS : Windows10
Python: 3.6.8
Tensorflow_GPU version: 1.8
GPU: GTX 1060 6 GB

Can you tell where the problem lies ? In the meantime, I'll try to run this the program with default configs on an Ubuntu machine and see the results. Thanks !

when I run the trainer, I got an error(UnknownError (see above for traceback): Fail to find the dnn implementation.)
tensorflow 1.5
python 2.7
ubuntu 16.04
please help me

The final softmax prob. values are not same if the padding amount changes. It looks like that for some of the functions such as reduce_mean and reduce_max the padding is not masked out. Also the word/char embeddings for zero padded tokens is not zero.

Hello,
I was told to try to run the program on Ubuntu 16.04 with python 2.7 and tensorFlow already installed, but when I tried to "python SentenceMatchTrainer.py", I got this error
.
Can anyone help me how to solve this problem?

Thanks

Could you please answer a quick question regarding evaluation on WikiQA?

When evaluating, did you eliminate only the questions which have no correct answers from DEV and TEST (thus having 126 DEV and 243 TEST questions)? Or did you eliminate the questions which have only correct answers as well (thus having 122 DEV and 237 TEST questions)?

Thank you!

Thank you for your work!
It not support class weight params to solve class imbalanced that leading to overfitted all the time.

This is the command I used to run the trainer:
SentenceMatchTrainer.py --train_path train.tsv --dev_path dev.tsv --test_path test.tsv --fix_word_vec --model_dir models/ --MP_dim 10 --suffix sample --word_vec_path wordvec.txt

And these are the training results so far:
Start the training loop.
0 100 200 300 400 500 600 700 800 900 1000 1100 1200 1300 1400 1500 1600 1700 1800 1900 2000 2100 2200 2300 2400 2500 2600 2700 2800 2900 3000 3100 3200 3300 3400 3500 3600 3700 3800 3900 4000 4100 4200 4300 4400 4500 4600 4700 4800 4900 5000 5100 5200 5300 5400 5500 5600 5700 5800 5900 6000 6100 6200 6300 6400
Step 6405: loss = 3255.40 (81732.544 sec)
Validation Data Eval:
Current accuracy is 79.97
6500 6600 6700 6800 6900 7000 7100 7200 7300 7400 7500 7600 7700 7800 7900 8000 8100 8200 8300 8400 8500 8600 8700 8800 8900 9000 9100 9200 9300 9400 9500 9600 9700 9800 9900 10000 10100 10200 10300 10400 10500 10600 10700 10800 10900 11000 11100 11200 11300 11400 11500 11600 11700 11800 11900 12000 12100 12200 12300 12400 12500 12600 12700 12800
Step 12811: loss = 2481.54 (81754.701 sec)
Validation Data Eval:
Current accuracy is 83.78
12900 13000 13100 13200 13300 13400 13500 13600 13700 13800 13900 14000 14100 14200 14300 14400 14500 14600 14700 14800 14900 15000 15100 15200 15300 15400 15500 15600 15700 15800 15900 16000 16100 16200 16300 16400 16500 16600 16700 16800 16900 17000 17100 17200 17300 17400 17500 17600 17700 17800 17900 18000 18100 18200 18300 18400 18500 18600 18700 18800 18900 19000 19100 19200
Step 19217: loss = 2204.13 (28593.611 sec)
Validation Data Eval:
Current accuracy is 84.22
19300 19400 19500 19600 19700 19800 19900 20000 20100 20200 20300 20400 20500 20600 20700 20800 20900 21000 21100 21200 21300 21400 21500 21600 21700 21800 21900 22000 22100 22200 22300 22400 22500 22600 22700 22800 22900 23000 23100 23200 23300 23400 23500 23600 23700 23800 23900 24000 24100 24200 24300 24400 24500 24600 24700 24800 24900 25000 25100 25200 25300 25400 25500 25600
Step 25623: loss = 2029.37 (296247.130 sec)
Validation Data Eval:
Current accuracy is 85.67
25700 25800 25900 26000 26100 26200 26300 26400 26500 26600 26700 26800 26900 27000 27100 27200 27300

Will it make 10 rounds of this because I set the --MP_dim parameter to 10? Or am I missing something?

TypeError Traceback (most recent call last)
in ()
251 FLAGS = enrich_options(FLAGS)
252
--> 253 main(FLAGS)
254

in main(FLAGS)
173 with tf.variable_scope("Model", reuse=None, initializer=initializer):
174 train_graph = SentenceMatchModelGraph(num_classes, word_vocab=word_vocab, char_vocab=char_vocab,
--> 175 is_training=True, options=FLAGS, global_step=global_step)
176
177 with tf.variable_scope("Model", reuse=True, initializer=initializer):

/notebooks/BiMPM1/src/SentenceMatchModelGraph.pyc in init(self, num_classes, word_vocab, char_vocab, is_training, options, global_step)
8 self.options = options
9 self.create_placeholders()
---> 10 self.create_model_graph(num_classes, word_vocab, char_vocab, is_training, global_step=global_step)
11
12 def create_placeholders(self):

/notebooks/BiMPM1/src/SentenceMatchModelGraph.pyc in create_model_graph(self, num_classes, word_vocab, char_vocab, is_training, global_step)
95 (question_char_outputs_fw, question_char_outputs_bw, _) = layer_utils.my_lstm_layer(in_question_char_repres, options.char_lstm_dim,
96 input_lengths=question_char_lengths,scope_name="char_lstm", reuse=False,
---> 97 is_training=is_training, dropout_rate=options.dropout_rate, use_cudnn=options.use_cudnn)
98 question_char_outputs_fw = layer_utils.collect_final_step_of_lstm(question_char_outputs_fw, question_char_lengths - 1)
99 question_char_outputs_bw = question_char_outputs_bw[:, 0, :]

/notebooks/BiMPM1/src/layer_utils.py in my_lstm_layer(input_reps, lstm_dim, input_lengths, scope_name, reuse, is_training, dropout_rate, use_cudnn)
18 inputs = tf.transpose(input_reps, [1, 0, 2])
19 lstm = tf.contrib.cudnn_rnn.CudnnLSTM(1, lstm_dim, direction="bidirectional",
---> 20 name="{}_cudnn_bi_lstm".format(scope_name), dropout=dropout_rate if is_training else 0)
21 outputs, _ = lstm(inputs)
22 outputs = tf.transpose(outputs, [1, 0, 2])

TypeError: init() got an unexpected keyword argument 'name'

Thanks very much！

I'm curious about this code is possible to check user defined sentence and matched closed match sentence like retrieval based chat bot . In my use case I want to train my model with some questions and check against user defined question to find out closest match question.

Could you please explain the meaning of MP_dim, decompose_params in the matching layer, why do you need them? These confuse me a lot.
Thanks.

The papers mentions that glove embeddings were used for word representation layer. However, when I tried to train with glove.840B.300d.txt I got the following error:

Cannot create a tensor proto whose content is larger than 2GB.

What preprocessing is applied for obtaining wordvec?

I wonder how to train the wordvec file you offered in the google drive, is it trained with the SNLI/Quora dataset corpus?

Could you please also share the configuration file for the WikiQA and TrecQA experiments to achieve your best results in the paper? Thank you very much!

Hi, when i try to run your code on the GPU , I got an error as follow:
2018-05-14 20:28:33.623427: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Traceback (most recent call last):
File "SentenceMatchTrainer.py", line 263, in
main(FLAGS)
File "SentenceMatchTrainer.py", line 201, in main
sess.run(initializer)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1128, in _run
feed_dict_tensor, options, run_metadata)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1344, in _do_run
options, run_metadata)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1363, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device for operation 'Model/word_embedding': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
[[Node: Model/word_embedding = VariableV2container="", dtype=DT_FLOAT, shape=[35611,300], shared_name="", _device="/device:GPU:0"]]

Caused by op u'Model/word_embedding', defined at:
File "SentenceMatchTrainer.py", line 263, in
main(FLAGS)
File "SentenceMatchTrainer.py", line 185, in main
is_training=True, options=FLAGS, global_step=global_step)
File "/home/chunlin/exp/BiMPM_SNLI/src/SentenceMatchModelGraph.py", line 10, in init
self.create_model_graph(num_classes, word_vocab, char_vocab, is_training, global_step=global_step)
File "/home/chunlin/exp/BiMPM_SNLI/src/SentenceMatchModelGraph.py", line 57, in create_model_graph
initializer=tf.constant(word_vocab.word_vecs), dtype=tf.float32)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1262, in get_variable
constraint=constraint)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1097, in get_variable
constraint=constraint)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 435, in get_variable
constraint=constraint)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 404, in _true_getter
use_resource=use_resource, constraint=constraint)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 806, in _get_single_variable
constraint=constraint)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 229, in init
constraint=constraint)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 351, in _init_from_args
name=name)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 133, in variable_op_v2
shared_name=shared_name)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 1061, in _variable_v2
shared_name=shared_name, name=name)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3160, in create_op
op_def=op_def)
File "/home/chunlin/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1625, in init
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Cannot assign a device for operation 'Model/word_embedding': Operation was explicitly assigned to /device:GPU:0 but available devices are [ /job:localhost/replica:0/task:0/device:CPU:0 ]. Make sure the device specification refers to a valid device.
[[Node: Model/word_embedding = VariableV2container="", dtype=DT_FLOAT, shape=[35611,300], shared_name="", _device="/device:GPU:0"]]

could you please tell me why this happened?

@zhiguowang Thanks a lot for publishing the algorithm and code of BiMPM. I am implementing the BiMPM algorithm in the AllenNLP library so that more people could experiment and benefit from this algorithm. Please check the PR allenai/allennlp#1594 for the re-implementation. It is based on your paper as well as the code here, and also can reproduce most of the results you described in the paper. Please help to review the code and let me know if you have any comments and suggestions about the PR. Thanks!

Where do we get the SentenceMatch.best.model file from? It is not being generated.
Accuracy coming out to be 0.

I'm using Glove pre-trained vectors (840B, 300D, Common Crawl).
When I try to train the system, I get the following error:

Configurations:
Namespace(MP_dim=20, NER_dim=20, POS_dim=20, aggregation_layer_num=1, aggregation_lstm_dim=100, batch_size=60, char_emb_dim=20, char_lstm_dim=100, context_layer_num=1, context_lstm_dim=100, dev_path='./dev.tsv', dropout_rate=0.1, fix_word_vec=True, highway_layer_num=1, lambda_l2=0.0, learning_rate=0.001, lex_decompsition_dim=-1, max_char_per_word=10, max_epochs=10, max_sent_length=100, model_dir='models', optimize_type='adam', suffix='sample', test_path='./test.tsv', train_path='./train.tsv', with_NER=False, with_POS=False, with_aggregation_highway=False, with_filter_layer=False, with_highway=False, with_lex_decomposition=False, with_match_highway=False, wo_attentive_match=False, wo_char=False, wo_full_match=False, wo_left_match=False, wo_max_attentive_match=False, wo_maxpool_match=False, wo_right_match=False, word_level_MP_dim=-1, word_vec_path='./glove.840B.300d.txt')
Traceback (most recent call last):
  File "src/SentenceMatchTrainer.py", line 391, in <module>
    tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
  File "/home/syed.b/stanford/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
    sys.exit(main(sys.argv[:1] + flags_passthrough))
  File "src/SentenceMatchTrainer.py", line 128, in main
    word_vocab = Vocab(word_vec_path, fileformat='txt3')
  File "/home/syed.b/Research/BiMPM/src/vocab_utils.py", line 18, in __init__
    self.fromText_format3(vec_path,voc=voc)
  File "/home/syed.b/Research/BiMPM/src/vocab_utils.py", line 143, in fromText_format3
    self.word_vecs[cur_index] = word_vecs[cur_index]
ValueError: could not broadcast input array from shape (299) into shape (300)

The format of the Glove txt file I'm using is:
< word > < 1st dim.value > < 2nd dim. value > ..... < 300th dim. value >
(without the relational signs of course)

When I load the file with many data, I have met with a problem. The free memory will be smaller and smaller because of the exitence of sorting algorithm in the preprocessing step. What should I do to optimize it ?

Hi, I like your work, and your code is decent. But I'm a little confused about the tensor calculation in matching. In match_utils.py, function multi_perspective_match:

mp_cosine_matching = layer_utils.cosine_distance(tf.multiply(repres1_flat, mp_cosine_params),
                                                             repres2_flat, cosine_norm=False)

Why only tf.multiply(repres1_flat, mp_cosine_params) , but not tf.multiply(repres2_flat, mp_cosine_params) as well ? In your paper, W is multiplied by both v1 and v2.

Looking forward to reply, thanks!

I wonder how many epochs you run on Quora dataset?

Hello,

The model was finally done with training on the dev set but when it was about to start with the test set, it gave a really big error:

Validation Data Eval:
Current accuracy is 87.35
Best accuracy on dev set is 87.35
Decoding on the test set:
WARNING:tensorflow:From SentenceMatchTrainer.py:330 in main.: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Please use tf.global_variables instead.
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
W tensorflow/core/framework/op_kernel.cc:975] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
Traceback (most recent call last):
File "SentenceMatchTrainer.py", line 391, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "SentenceMatchTrainer.py", line 339, in main
saver.restore(sess, best_path)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1388, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Caused by op u'save/RestoreV2', defined at:
File "SentenceMatchTrainer.py", line 391, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "SentenceMatchTrainer.py", line 334, in main
saver = tf.train.Saver(vars_)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1000, in init
self.build()
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1030, in build
restore_sequentially=self._restore_sequentially)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 624, in build
restore_sequentially, reshape)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 361, in _AddRestoreOps
tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 200, in restore_op
[spec.tensor.dtype])[0])
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 441, in restore_v2
dtypes=dtypes, name=name)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/Users/dc-hadil/tensorflow/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1128, in init
self._traceback = _extract_stack()

NotFoundError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to find any matching files for models//SentenceMatch.sample.best.model
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Is there a way to pick up the process from the test set? or do I have to start with the training all over again? Also, why was there an error?

Hello, I am interested to this model and want to know its training time of one epoch in SNLI or Quora dataset. And how many epoches does it need to reach the convergence state?