currie32 / chatbot-from-movie-dialogue Goto Github PK
View Code? Open in Web Editor NEWBuilt a simple chatbot from a sequence-to-sequence model with TensorFlow.
Built a simple chatbot from a sequence-to-sequence model with TensorFlow.
In GitHub you have been using 'process_decoding_input' but in blog spot you have mentioned 'process_encoding_input' for the same method which one is correct you please let us know?
I am using ubuntu 16.04
version of tensorflow is 1.3.0
installed and validated using anaconda
While trying to create training and inference logits of input[32] of the following python notebook
https://github.com/Currie32/Chatbot-from-Movie-Dialogue/blob/master/Chatbot_Attention.ipynb
I was getting errors while executing the following command:
train_logits, inference_logits = seq2seq_model( tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(answers_vocab_to_int), len(questions_vocab_to_int), encoding_embedding_size, decoding_embedding_size, rnn_size, num_layers, questions_vocab_to_int)
Stack trace:
Traceback (most recent call last):
File "", line 4, in
File "", line 6, in seq2seq_model
File "", line 10, in encoding_layer
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 405, in bidirectional_dynamic_rnn
time_major=time_major, scope=fw_scope)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 598, in dynamic_rnn
dtype=dtype)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 761, in _dynamic_rnn_loop
swap_memory=swap_memory)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2775, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2604, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2554, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 744, in _time_step
skip_conditionals=True)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 236, in _rnn_step
new_output, new_state = call_cell()
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 732, in
call_cell = lambda: cell(input_t, state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 180, in call
return super(RNNCell, self).call(inputs, state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 450, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 938, in call
cur_inp, new_state = cell(cur_inp, cur_state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 774, in call
output, new_state = self._cell(inputs, state, scope)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 180, in call
return super(RNNCell, self).call(inputs, state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 450, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 401, in call
concat = _linear([inputs, h], 4 * self._num_units, True)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1039, in _linear
initializer=kernel_initializer)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1065, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 962, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 360, in get_variable
validate_shape=validate_shape, use_resource=use_resource)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1405, in wrapped_custom_getter
*args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in _rnn_get_variable
variable = getter(*args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in _rnn_get_variable
variable = getter(*args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 352, in _true_getter
use_resource=use_resource)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 664, in _get_single_variable
name, "".join(traceback.format_list(tb))))
ValueError: Variable bidirectional_rnn/fw/multi_rnn_cell/cell_0/basic_lstm_cell/kernel already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:
File "", line 10, in encoding_layer
File "", line 6, in seq2seq_model
File "", line 2, in
I am using ubuntu 16.04
version of tensorflow is 1.3.0
installed and validated using anaconda
While trying to create training and inference logits of input[32] of the following python notebook
https://github.com/Currie32/Chatbot-from-Movie-Dialogue/blob/master/Chatbot_Attention.ipynb
I was getting errors while executing the following command:
train_logits, inference_logits = seq2seq_model( tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(answers_vocab_to_int), len(questions_vocab_to_int), encoding_embedding_size, decoding_embedding_size, rnn_size, num_layers, questions_vocab_to_int)
Stack trace:
Traceback (most recent call last):
File "", line 4, in
File "", line 6, in seq2seq_model
File "", line 10, in encoding_layer
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 405, in bidirectional_dynamic_rnn
time_major=time_major, scope=fw_scope)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 598, in dynamic_rnn
dtype=dtype)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 761, in _dynamic_rnn_loop
swap_memory=swap_memory)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2775, in while_loop
result = context.BuildLoop(cond, body, loop_vars, shape_invariants)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2604, in BuildLoop
pred, body, original_loop_vars, loop_vars, shape_invariants)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/control_flow_ops.py", line 2554, in _BuildLoop
body_result = body(*packed_vars_for_body)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 744, in _time_step
skip_conditionals=True)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 236, in _rnn_step
new_output, new_state = call_cell()
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn.py", line 732, in
call_cell = lambda: cell(input_t, state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 180, in call
return super(RNNCell, self).call(inputs, state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 450, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 938, in call
cur_inp, new_state = cell(cur_inp, cur_state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 774, in call
output, new_state = self._cell(inputs, state, scope)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 180, in call
return super(RNNCell, self).call(inputs, state)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/layers/base.py", line 450, in call
outputs = self.call(inputs, *args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 401, in call
concat = _linear([inputs, h], 4 * self._num_units, True)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 1039, in _linear
initializer=kernel_initializer)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1065, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 962, in get_variable
use_resource=use_resource, custom_getter=custom_getter)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 360, in get_variable
validate_shape=validate_shape, use_resource=use_resource)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 1405, in wrapped_custom_getter
*args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in _rnn_get_variable
variable = getter(*args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/rnn_cell_impl.py", line 183, in _rnn_get_variable
variable = getter(*args, **kwargs)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 352, in _true_getter
use_resource=use_resource)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/ops/variable_scope.py", line 664, in _get_single_variable
name, "".join(traceback.format_list(tb))))
ValueError: Variable bidirectional_rnn/fw/multi_rnn_cell/cell_0/basic_lstm_cell/kernel already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:
File "", line 10, in encoding_layer
File "", line 6, in seq2seq_model
File "", line 2, in
Hi,
This is the issue I'm getting after running. Here's the part of the code in which I'm getting issues. I'm using python 3.5 in anaconda and spyder. One more thing, I know I'm using BahdanauAttention, as attention_option was giving error so I changed it to this and then also I was facing many errors so I referred to the tensorflow's guide and found out this and this actually work. But with a small error.
File "E:/CHATBOT/CHATBOT/Chatbot-from-Movie-Dialogue-master/Chatbot_Attention.py", line 439, in decoding_layer_train
num_units=dec_cell.output_size)
def decoding_layer_train(encoder_state, dec_cell, dec_embed_input, sequence_length, decoding_scope,
output_fn, keep_prob, batch_size):
'''Decode the training data'''
attention_states = tf.zeros([batch_size, 1, dec_cell.output_size])
att_keys, att_vals, att_score_fn, att_construct_fn = tf.contrib.seq2seq.BahdanauAttention(attention_states,
name="bahdanau",
num_units=dec_cell.output_size)
train_decoder_fn = tf.contrib.seq2seq.attention_decoder_fn_train(encoder_state[0],
att_keys,
att_vals,
att_score_fn,
att_construct_fn,
name = "attn_dec_train")
train_pred, _, _ = tf.contrib.seq2seq.dynamic_rnn_decoder(dec_cell,
train_decoder_fn,
dec_embed_input,
sequence_length,
scope=decoding_scope)
train_pred_drop = tf.nn.dropout(train_pred, keep_prob)
return output_fn(train_pred_drop)
I am able to save the model properly using line but while restoring it, I am getting an error:
Traceback (most recent call last):
File "start_something.py", line 254, in <module>
saver = tf.train.import_meta_graph('/media/saurabh/New Volume/nlp_ucf/project/model/model_0.ckpt.meta')
File "/home/saurabh/tfenv3/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1577, in import_meta_graph
**kwargs)
File "/home/saurabh/tfenv3/lib/python3.5/site-packages/tensorflow/python/framework/meta_graph.py", line 498, in import_scoped_meta_graph
producer_op_list=producer_op_list)
File "/home/saurabh/tfenv3/lib/python3.5/site-packages/tensorflow/python/framework/importer.py", line 259, in import_graph_def
raise ValueError('No op named %s in defined operations.' % node.op)
ValueError: No op named attn_add_fun_f32f32f32 in defined operations.
Any idea if you got the same error? You don't restore the model in your code at all.
Thanks for the help.
I am training the model and have the weights written out to checkpoint. I am not able to load the weights to test the model. Anyone have advice on how to do so?
I set epoch=1, and after training model how can I test the code? here is the output:
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE3 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
117585
20750
Epoch 1/1 Batch 0/918 - Loss: 0.090, Seconds: 341.72
Epoch 1/1 Batch 100/918 - Loss: 3.188, Seconds: 359.53
Epoch 1/1 Batch 200/918 - Loss: 2.284, Seconds: 426.48
Epoch 1/1 Batch 300/918 - Loss: 2.204, Seconds: 422.31
Epoch 1/1 Batch 400/918 - Loss: 2.125, Seconds: 422.03
Valid Loss: 2.075, Seconds: 188.65
New Record!
Epoch 1/1 Batch 500/918 - Loss: 2.099, Seconds: 534.29
Epoch 1/1 Batch 600/918 - Loss: 2.116, Seconds: 769.90
Epoch 1/1 Batch 700/918 - Loss: 2.097, Seconds: 556.36
Epoch 1/1 Batch 800/918 - Loss: 2.081, Seconds: 622.04
Epoch 1/1 Batch 900/918 - Loss: 2.040, Seconds: 735.49
Valid Loss: 2.050, Seconds: 173.22
New Record!
Question
Word Ids: [5610, 3781, 2805, 7279, 1926]
Input Words: ['irene', 'is', 'going', 'to', 'help']
Answer
Word Ids: [4776, 951, 7075, 8094]
Response Words: ['i', 'am', 'sorry', '<EOS>']
as@asd ~/c/Chatbot-from-Movie-Dialogue>
how can I test conversation with the bot?
This part of the code:
# Create the training and inference logits
train_logits, inference_logits = seq2seq_model(
tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(answers_vocab_to_int),
len(questions_vocab_to_int), encoding_embedding_size, decoding_embedding_size, rnn_size, num_layers,
questions_vocab_to_int)
Fails with the following error. Anyone get the code to work with TF 1.2.1 or 1.3?
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-42-5eb2c4ab2c25> in <module>()
15 tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(answers_vocab_to_int),
16 len(questions_vocab_to_int), encoding_embedding_size, decoding_embedding_size, rnn_size, num_layers,
---> 17 questions_vocab_to_int)
18
19 # Create a tensor for the inference logits, needed if loading a checkpoint version of the model
<ipython-input-39-bbac5bbc5884> in seq2seq_model(input_data, target_data, keep_prob, batch_size, sequence_length, answers_vocab_size, questions_vocab_size, enc_embedding_size, dec_embedding_size, rnn_size, num_layers, questions_vocab_to_int)
24 questions_vocab_to_int,
25 keep_prob,
---> 26 batch_size)
27 return train_logits, infer_logits
<ipython-input-38-4c62787c7f16> in decoding_layer(dec_embed_input, dec_embeddings, encoder_state, vocab_size, sequence_length, rnn_size, num_layers, vocab_to_int, keep_prob, batch_size)
24 output_fn,
25 keep_prob,
---> 26 batch_size)
27 decoding_scope.reuse_variables()
28 infer_logits = decoding_layer_infer(encoder_state,
<ipython-input-36-c7b11c624372> in decoding_layer_train(encoder_state, dec_cell, dec_embed_input, sequence_length, decoding_scope, output_fn, keep_prob, batch_size)
5 attention_states = tf.zeros([batch_size, 1, dec_cell.output_size])
6
----> 7 att_keys, att_vals, att_score_fn, att_construct_fn = tf.contrib.seq2seq.prepare_attention(attention_states,
8 attention_option="bahdanau",
9 num_units=dec_cell.output_size)
AttributeError: module 'tensorflow.contrib.seq2seq' has no attribute 'prepare_attention'
Question
Word Ids: [84, 17, 11, 145, 223]
Input Words: ['what', 'are', 'you', 'doing', 'here']
Answer
Word Ids: [1, 85, 12, 2202, 2201]
Response Words: ['i', 'am', 'not', '', '']
Question
Word Ids: [849, 2154, 49, 12, 2202, 36, 1, 85, 18, 11, 16, 12, 52, 2202, 36]
Input Words: ['mrs', 'robinson', 'do', 'not', '', 'it', 'i', 'am', 'asking', 'you', 'please', 'not', 'to', '', 'it']
Answer
Word Ids: [1, 85, 12, 2202, 2201]
Response Words: ['i', 'am', 'not', '', '']
I have trained the model once but it gives same answer for all the questions , so I had to delete all the trained data and then again I had to train to get different answers.
question:what are you doing
word ids: [7906,1941,8103]
input words :['what','are','you','doing']
answer:
word ids:[7556,7556,548,548,548,345,345,345,345,987,987,987]
output words:['kick','kick','within','within','within','test','test,'test','test','process','process','process']
why I get like this,where i went wrong
Please improve your documentation. this is irritating.
and let me know the solution for "TypeError: Expected bool passed to parameter 'dims' of op 'Reverse', got [-1] of type 'list' instead."
I am using python 3.5 version and tensorflow-1.0.0 version. I have installed tensorflow with CPU support only.
C:\Chatbot-from-Movie-Dialogue-master>python Chatbot_Attention.py
Can we make this quick? Roxanne Korrine and Andrew Barrett are having an incredibly horrendous public break- up on the quad. Again.
Well, I thought we'd start with pronunciation, if that's okay with you.
Well, I thought we'd start with pronunciation, if that's okay with you.
Not the hacking and gagging and spitting part. Please.
Not the hacking and gagging and spitting part. Please.
Okay... then how 'bout we try out some French cuisine. Saturday? Night?
You're asking me out. That's so cute. What's your name again?
Forget it.
No, no, it's my fault -- we didn't have a proper introduction ---
Cameron.
221616
221616
can we make this quick roxanne korrine and andrew barrett are having an incredibly horrendous public break up on the quad again
well i thought we would start with pronunciation if that is okay with you
well i thought we would start with pronunciation if that is okay with you
not the hacking and gagging and spitting part please
not the hacking and gagging and spitting part please
okay then how about we try out some french cuisine saturday night
you are asking me out that is so cute that is your name again
forget it
no no it is my fault we did not have a proper introduction
cameron
16.0
19.0
24.0
32.0
58.0
% of data used: 62.419999999999995%
Size of total vocab: 45618
Size of vocab we will use: 8092
8096
8096
8096
8096
138335
138335
Total number of words: 2334533
Number of times is used: 92436
Percent of words that are : 3.96%
138335
138335
[7645, 2615]
[5230, 4733, 4733, 4733, 2740, 6471, 2991, 5230, 3754, 3491, 647, 6277, 4990, 5839, 8094]
[5230, 4959]
[1211, 7456, 7793, 5378, 6471, 7662, 307, 8095, 5121, 1211, 8094]
[2465, 2693]
[1331, 590, 7162, 4811, 1211, 1835, 5378, 1322, 8094]
Traceback (most recent call last):
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 490, in apply_op
preferred_dtype=default_dtype)
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\ops.py", line 669, in convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\constant_op.py", line 176, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\constant_op.py", line 165, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 367, in make_tensor_proto
_AssertCompatible(values, dtype)
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).name))
TypeError: Expected bool, got -1 of type 'int' instead.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Chatbot_Attention.py", line 591, in
tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(answers_vocab_to_int),
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 2513, in reverse
name=name)
File "C:\Users\Apurva\AppData\Local\Programs\Python\Python35\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 499, in apply_op
repr(values), type(values).name))
TypeError: Expected bool passed to parameter 'dims' of op 'Reverse', got [-1] of type 'list' instead.
C:\Chatbot-from-Movie-Dialogue-master>
Hi,
I am getting the error 'ValueError: lengths must be 1D for sequence_mask' when I run this part of the code:
train_logits, inference_logits = seq2seq_model(
tf.reverse(input_data, [-1]), targets, keep_prob, batch_size, sequence_length, len(answers_vocab_to_int),
len(questions_vocab_to_int), encoding_embedding_size, decoding_embedding_size, rnn_size, num_layers, questions_vocab_to_int)
Can anyone help me in resolving this?
############# Buildimg the Sec2Sec model #########
def model_inputs():
inputs = tf.placeholder(tf.int32, [None, None], name = 'input')
targets = tf.placeholder(tf.int32, [None, None], name = 'target')
lr = tf.placeholder(tf.float32, name = 'learning_rate')
keep_prob = tf.placeholder(tf.float32, name = 'keep_prob')
return inputs, targets, lr, keep_prob
def preprocess_targets(targets, word2int, batch_size):
left_side = tf.fill([batch_size, 1], word2int[''])
right_side = tf.strided_slice(targets, [0,0], [batch_size, -1], [1,1])
preprocessed_targets = tf.concat([left_side, right_side], 1)
return preprocessed_targets
def encoder_rnn(rnn_inputs, rnn_size, num_layers, keep_prob, sequence_length):
lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)
lstm_dropout = tf.contrib.rnn.DropoutWrapper(lstm, input_keep_prob = keep_prob)
encoder_cell = tf.contrib.rnn.MultiRNNCell([lstm_dropout] * num_layers)
encoder_output, encoder_state = tf.nn.bidirectional_dynamic_rnn(cell_fw = encoder_cell,
cell_bw = encoder_cell,
sequence_length = sequence_length,
inputs = rnn_inputs,
dtype = tf.float32)
def decode_training_set(encoder_state, decoder_cell, decoder_embedded_input, sequence_length, decoding_scope, output_function, keep_prob, batch_size):
attention_states = tf.zeros([batch_size, 1, decoder_cell.output_size])
attention_keys, attention_values, attention_score_function, attention_construct_function = tf.contrib.seq2seq.prepare_attention(attention_states, attention_option = "bahdanau", num_units = decoder_cell.output_size)
training_decoder_function = tf.contrib.seq2seq.attention_decoder_fn_train(encoder_state[0],
attention_keys,
attention_values,
attention_score_function,
attention_construct_function,
name = "attn_dec_train")
decoder_output, decoder_final_state, decoder_final_context_state = tf.contrib.seq2seq.dynamic_rnn_decoder(decoder_cell,
training_decoder_function,
decoder_embedded_input,
sequence_length,
scope = decoding_scope)
decoder_output_dropout = tf.nn.dropout(decoder_output, keep_prob)
return output_function(decoder_output_dropout)
def decode_test_set(encoder_state, decoder_cell, decoder_embeddings_matrix, sos_id, eos_id, maximum_length, num_words, decoding_scope, output_function, keep_prob, batch_size):
attention_states = tf.zeros([batch_size, 1, decoder_cell.output_size])
attention_keys, attention_values, attention_score_function, attention_construct_function = tf.contrib.seq2seq.prepare_attention(attention_states, attention_option = "bahdanau", num_units = decoder_cell.output_size)
test_decoder_function = tf.contrib.seq2seq.attention_decoder_fn_inference(output_function,
encoder_state[0],
attention_keys,
attention_values,
attention_score_function,
attention_construct_function,
decoder_embeddings_matrix,
sos_id,
eos_id,
maximum_length,
num_words,
name = "attn_dec_inf")
test_predictions, decoder_final_state, decoder_final_context_state = tf.contrib.seq2seq.dynamic_rnn_decoder(decoder_cell,
test_decoder_function,
scope = decoding_scope)
return test_predictions
def decoder_rnn(decoder_embedded_input, decoder_embeddings_matrix, encoder_state, num_words, sequence_length, rnn_size, num_layers, word2int, keep_prob, batch_size):
with tf.variable_scope("decoding") as decoding_scope:
lstm = tf.contrib.rnn.BasicLSTMCell(rnn_size)
lstm_dropout = tf.contrib.rnn.DropoutWrapper(lstm, input_keep_prob = keep_prob)
decoder_cell = tf.contrib.rnn.MultiRNNCell([lstm_dropout] * num_layers)
weights = tf.truncated_normal_initializer(stddev = 0.1)
biases = tf.zeros_initializer()
output_function = lambda x: tf.contrib.layers.fully_connected(x,
num_words,
None,
scope = decoding_scope,
weights_initializer = weights,
biases_initializer = biases)
training_predictions = decode_training_set(encoder_state,
decoder_cell,
decoder_embedded_input,
sequence_length,
decoding_scope,
output_function,
keep_prob,
batch_size)
decoding_scope.reuse_variables()
test_predictions = decode_test_set(encoder_state,
decoder_cell,
decoder_embeddings_matrix,
word2int[''],
word2int[''],
sequence_length - 1,
num_words,
decoding_scope,
output_function,
keep_prob,
batch_size)
return training_predictions, test_predictions
def seq2seq_model(inputs, targets, keep_prob, batch_size, sequence_length, answers_num_words, questions_num_words, encoder_embedding_size, decoder_embedding_size, rnn_size, num_layers, questionswords2int):
encoder_embedded_input = tf.contrib.layers.embed_sequence(inputs,
answers_num_words + 1,
encoder_embedding_size,
initializer = tf.random_uniform_initializer(0, 1))
encoder_state = encoder_rnn(encoder_embedded_input, rnn_size, num_layers, keep_prob, sequence_length)
preprocessed_targets = preprocess_targets(targets, questionswords2int, batch_size)
decoder_embeddings_matrix = tf.Variable(tf.random_uniform([questions_num_words + 1, decoder_embedding_size], 0, 1))
decoder_embedded_input = tf.nn.embedding_lookup(decoder_embeddings_matrix, preprocessed_targets)
training_predictions, test_predictions = decoder_rnn(decoder_embedded_input,
decoder_embeddings_matrix,
encoder_state,
questions_num_words,
sequence_length,
rnn_size,
num_layers,
questionswords2int,
keep_prob,
batch_size)
return training_predictions, test_predictions
epochs = 100
batch_size = 64
rnn_size = 512
num_layers = 3
encoding_embedding_size = 512
decoding_embedding_size = 512
learning_rate = 0.01
learning_name_decay = 0.9
min_learning_rate = 0.0001
keep_probability = 0.5
tf.reset_default_graph()
session = tf.InteractiveSession()
inputs, targets, lr, keep_prob = model_inputs()
sequence_length = tf.placeholder_with_default(25, None, name = 'sequence_length')
input_shape = tf.shape(inputs)
training_predictions, test_predictions = seq2seq_model(tf.reverse(inputs, [-1]),
targets,
keep_prob,
batch_size,
sequence_length,
len(answerswords2int),
len(questionswords2int),
encoding_embedding_size,
decoding_embedding_size,
rnn_size,
num_layers,
questionswords2int)
TypeError Traceback (most recent call last)
in ()
12 rnn_size,
13 num_layers,
---> 14 questionswords2int)
in seq2seq_model(inputs, targets, keep_prob, batch_size, sequence_length, answers_num_words, questions_num_words, encoder_embedding_size, decoder_embedding_size, rnn_size, num_layers, questionswords2int)
19 questionswords2int,
20 keep_prob,
---> 21 batch_size)
22 return training_predictions, test_predictions
in decoder_rnn(decoder_embedded_input, decoder_embeddings_matrix, encoder_state, num_words, sequence_length, rnn_size, num_layers, word2int, keep_prob, batch_size)
21 output_function,
22 keep_prob,
---> 23 batch_size)
24 decoding_scope.reuse_variables()
25 test_predictions = decode_test_set(encoder_state,
in decode_training_set(encoder_state, decoder_cell, decoder_embedded_input, sequence_length, decoding_scope, output_function, keep_prob, batch_size)
4 attention_states = tf.zeros([batch_size, 1, decoder_cell.output_size])
5 attention_keys, attention_values, attention_score_function, attention_construct_function = tf.contrib.seq2seq.prepare_attention(attention_states, attention_option = "bahdanau", num_units = decoder_cell.output_size)
----> 6 training_decoder_function = tf.contrib.seq2seq.attention_decoder_fn_train(encoder_state[0],
7 attention_keys,
8 attention_values,
TypeError: 'NoneType' object is not subscriptable
Is possible to modify script for getting user input and show display results. Now it take only one input for testing and for one single input need to run whole epoch step iteration. Is possible to run epoch one time and load model and test with user given input ?
thanks for the code
Though i couldn't understand the purpose of removing questions and answers less than 2 words.
Please explain
`# Remove questions and answers that are shorter than 2 words and longer than 20 words.
min_line_length = 2
max_line_length = 20
short_questions_temp = []
short_answers_temp = []
i = 0
for question in clean_questions:
if len(question.split()) >= min_line_length and len(question.split()) <= max_line_length:
short_questions_temp.append(question)
short_answers_temp.append(clean_answers[i])
i += 1
short_questions = []
short_answers = []
i = 0
for answer in short_answers_temp:
if len(answer.split()) >= min_line_length and len(answer.split()) <= max_line_length:
short_answers.append(answer)
short_questions.append(short_questions_temp[i])
i += 1`
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.