hunkim / word-rnn-tensorflow Goto Github PK
View Code? Open in Web Editor NEWMulti-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow.
License: MIT License
Multi-layer Recurrent Neural Networks (LSTM, RNN) for word-level language models in Python using TensorFlow.
License: MIT License
Hi, does weighted_pick()
return random characters?
with sampling_type=1
, return(int(np.searchsorted(t, np.random.rand(1)*s)))
looks like giving a random choice out of the array t. And that's what I'm having right now. Is it intentional? Or am I doing something wrong?
Hi:
Training and standard sampling working perfectly in tensorflow 12.1 on 64 bit ubuntu with gpu.
Problem occurs when choosing "beam search pick" for sampling. After 15 minutes of trying to generate default 200 words the following warning appears:
/home/aaron/Desktop/word-rnn-tensorflow/beam.py:26: RuntimeWarning: divide by zero encountered in log
cand_scores = np.array(live_scores)[:, None] - np.log(self.probs)
25 minutes still no terminal output or redirect to text file.
Any ideas how I can fix this?
Thanks
We need to change native python IO to support Google Storage, gs://
.
Some hits are:
#python3 train.py
reading text file
Traceback (most recent call last):
File "train.py", line 134, in
main()
File "train.py", line 52, in main
train(args)
File "train.py", line 88, in train
merged = tf.summary.merge_all()
AttributeError: 'module' object has no attribute 'merge_all'
Running it I'm getting this error.
I've install tensor flow with :
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.11.0-cp34-cp34m-linux_x86_64.whl
First of all, it is a very good repo, thanks to all who contributed. Actually, this is not an issue, just a question: Did you use any word embedding (like GloVe, doc2vec or fasttext) for the input words? If so, how can we change that from one to another? If not, where should we add it and how can we organize the dependent parameters such as vocab?
Hi,
I tried to train a model on the minishakespear dataset. Then, when I run sample.py, it keeps giving me output like this:
Fame,
Fame,
Fame, full complete; How many it trust no darest to lour? Dive, A merriment fled on a month. Nurse: As he, brother, at the tale his exile, thou shalt thereto with them swarm on Richmond.
There are always three "Fame" at the beginning. Even when I change the num to 1, it gives me output like:
Fame,
Fame,
Fame, Lancaster,
And in the sample function of model.py, I see this,
ret = prime
#other code
pred = words[sample]
ret += ' ' + pred
word = pred
return ret
So, I think the prime string will also be included in the output, right?
Can you help me in understanding the output? Thank you.
When I use "--init_from", the count on my training starts back at 0 instead of whatever it happened to be. Is this expected?
Hi, we can see that you prepare 3 files from the data directory.
“input.txt” represents the training data
"vocab.pkl" represents the vocabulary list
And i don't understand what does the "data.npy" stands for?
Should we put the save directory under the data directory? Or at least, we need to check if the directory exists.
The subject says it all. I'm still learning how to work with TF so I'm unable to correct it myself at this point..
Hey, Thank you for this model. I'm a noob, trying to utilize this to push my thoughts on writing. I'd like to use this model to see what sort of word couplings it could produce to incorporate into my novel. Thanks for the help.
I am getting this error. Using python 2.7 with Tensorflow 1.2 on mac - Sierra.
Stevens-MacBook-Pro:char-rnn-tensorflow-master StevenOchs$ python train.py
Traceback (most recent call last):
File "train.py", line 10, in
from model import Model
File "/Users/StevenOchs/Desktop/char-rnn-tensorflow-master/model.py", line 3, in
from tensorflow.contrib import legacy_seq2seq
ImportError: cannot import name legacy_seq2seq
Any thoughts?
Need to refactor with state_is_tuple=True for TF 0.11?
cell = cell_fn(args.rnn_size, state_is_tuple=True)
self.cell = cell = rnn_cell.MultiRNNCell([cell] * args.num_layers, state_is_tuple=True)
feed = {model.input_data: x, model.targets: y}
for i, (c, h) in enumerate(model.initial_state):
feed[c] = state[I].c
feed[h] = state[I].h
Looking for a way to determine performance.
Perhaps it's possible to use perplexity?
Something similar to the following pull request:
sherjilozair/char-rnn-tensorflow#56
python train.py
Traceback (most recent call last):
File "train.py", line 11, in <module>
from model import Model
File "/home/user/word-rnn-tensorflow/model.py", line 3, in <module>
from tensorflow.contrib import legacy_seq2seq
ImportError: cannot import name legacy_seq2seq
i have tensorflow ok
The dataset generated from each run is added up to the old one instead of generating a new one for indepedent runs.
Exciting news!
https://github.com/tensorflow/tensorflow/releases/tag/v1.0.0-alpha
However, it includes some API breaking changes. Perhaps, we need to refactor our code.
Please feel free to test and send us PR for TF 1.0!
Pretty much what it says in the title. I've noticed that it occasionally saves the model, so I was wondering if it's safe for me to stop it and resume it later.
I would like to see if the model improves during training or is overfitting. To this end, I would like on the one hand to add accuracy as a metric besides loss, and a validation set to see the validation loss and accuracy. I am new to Tensorflow, is there any easy way to do it? Thanks!
Getting the following when running on OS X, Python 2.7.10. Any ideas?
Traceback (most recent call last):
File "train.py", line 11, in
from model import Model
File "/word-rnn-tensorflow-master/model.py", line 2, in
from tensorflow.models.rnn import rnn_cell
File "/tensorflow/lib/python2.7/site-packages/tensorflow/models/rnn/rnn_cell.py", line 21, in
raise ImportError("This module is deprecated. Use tf.nn.rnn_cell instead.")
ImportError: This module is deprecated. Use tf.nn.rnn_cell instead.
How can I use word vs char level models? I don't see params for it in the config/train file. Thanks!
PS: Can the word level models support pretrained word embeddings?
Currently, we are using weighted_pick to select outputs:
def weighted_pick(weights):
t = np.cumsum(weights)
s = np.sum(weights)
return(int(np.searchsorted(t, np.random.rand(1)*s)))
Should we also add beam search as an option?
See beam search implementations in Tensorflow tensorflow/tensorflow#654!
Thank you for this wonderful program. I use it with a few writings of the german philosopher
Friedrich Wilhelm Nietzsche.
To use the program 'Model.py' with Python 3.52, the following changes must be made (#changed Python 3.5):
import tensorflow as tf
# from tensorflow.models.rnn import tf.nn.rnn_cell
# from tensorflow.models.rnn import tf.nn.seq2seq
import numpy as np
class Model():
def __init__(self, args, infer=False):
self.args = args
if infer:
args.batch_size = 1
args.seq_length = 1
if args.model == 'rnn':
cell_fn = tf.nn.rnn_cell.BasicRNNCell #changed
elif args.model == 'gru':
cell_fn = tf.nn.rnn_cell.GRUCell #changed
elif args.model == 'lstm':
cell_fn = tf.nn.rnn_cell.BasicLSTMCell #changed
else:
raise Exception("model type not supported: {}".format(args.model))
cell = cell_fn(args.rnn_size)
self.cell = cell = tf.nn.rnn_cell.MultiRNNCell([cell] * args.num_layers) #changed
self.input_data = tf.placeholder(tf.int32, [args.batch_size, args.seq_length])
self.targets = tf.placeholder(tf.int32, [args.batch_size, args.seq_length])
self.initial_state = cell.zero_state(args.batch_size, tf.float32)
with tf.variable_scope('rnnlm'):
softmax_w = tf.get_variable("softmax_w", [args.rnn_size, args.vocab_size])
softmax_b = tf.get_variable("softmax_b", [args.vocab_size])
with tf.device("/cpu:0"):
embedding = tf.get_variable("embedding", [args.vocab_size, args.rnn_size])
inputs = tf.split(1, args.seq_length, tf.nn.embedding_lookup(embedding, self.input_data))
inputs = [tf.squeeze(input_, [1]) for input_ in inputs]
def loop(prev, _):
prev = tf.matmul(prev, softmax_w) + softmax_b
prev_symbol = tf.stop_gradient(tf.argmax(prev, 1))
return tf.nn.embedding_lookup(embedding, prev_symbol)
outputs, last_state = tf.nn.seq2seq.rnn_decoder(inputs, self.initial_state, cell, loop_function=loop if infer else None, scope='rnnlm') #changed
output = tf.reshape(tf.concat(1, outputs), [-1, args.rnn_size])
self.logits = tf.matmul(output, softmax_w) + softmax_b
self.probs = tf.nn.softmax(self.logits)
loss = tf.nn.seq2seq.sequence_loss_by_example([self.logits], #changed
[tf.reshape(self.targets, [-1])],
[tf.ones([args.batch_size * args.seq_length])],
args.vocab_size)
self.cost = tf.reduce_sum(loss) / args.batch_size / args.seq_length
self.final_state = last_state
self.lr = tf.Variable(0.0, trainable=False)
tvars = tf.trainable_variables()
grads, _ = tf.clip_by_global_norm(tf.gradients(self.cost, tvars),
args.grad_clip)
optimizer = tf.train.AdamOptimizer(self.lr)
self.train_op = optimizer.apply_gradients(list(zip(grads, tvars))) #changed Python 3.5
def sample(self, sess, words, vocab, num=200, prime='first all', sampling_type=1):
state = self.cell.zero_state(1, tf.float32).eval()
prime = list(vocab.keys())[2] #changed Python 3.5
print(prime) #changed Python 3.5
for word in [prime]:
print (word)
x = np.zeros((1, 1))
x[0, 0] = vocab[word]
feed = {self.input_data: x, self.initial_state:state}
[state] = sess.run([self.final_state], feed)
def weighted_pick(weights):
t = np.cumsum(weights)
s = np.sum(weights)
return(int(np.searchsorted(t, np.random.rand(1)*s)))
ret = prime
word = prime
for n in range(num):
x = np.zeros((1, 1))
x[0, 0] = vocab[word]
feed = {self.input_data: x, self.initial_state:state}
[probs, state] = sess.run([self.probs, self.final_state], feed)
p = probs[0]
if sampling_type == 0:
sample = np.argmax(p)
elif sampling_type == 2:
if word == '\n':
sample = weighted_pick(p)
else:
sample = np.argmax(p)
else: # sampling_type == 1 default:
sample = weighted_pick(p)
pred = words[sample]
ret += ' ' + pred
word = pred
return ret
Does the script support Unicode? I've been trying to set it up with Malayalam but not able to
Edit: nevermind, I got the embeddings projector to work, will submit a pull request to add this to the project, it's a really nice feature for exploring data .
Thanks so much for your code hunkim! It is very helpful!
Can I ask a quick question please? Am I right to think: within one batch, every time you feed model.initial_state: state
to model
, it overrides
self.initial_state = cell.zero_state(args.batch_size, tf.float32)
(because this zero_state
initialisation is also in model def__init__()
thus I am not too sure whether it can be overriden )?
Thanks!
Hello. Please help my with little problem. I want to write simple Rest API with Flask:
from flask import Flask
import numpy as np
import tensorflow as tf
import argparse
import time
import os
from six.moves import cPickle
from utils import TextLoader
from model import Model
app = Flask(__name__)
def sample(n, pick, width, prime):
save_dir = './models/tinyshakespeare'
sample = 1
with open(os.path.join(save_dir, 'config.pkl'), 'rb') as f:
saved_args = cPickle.load(f)
with open(os.path.join(save_dir, 'words_vocab.pkl'), 'rb') as f:
words, vocab = cPickle.load(f)
model = Model(saved_args, True)
with tf.Session() as sess:
tf.global_variables_initializer().run()
saver = tf.train.Saver(tf.global_variables())
ckpt = tf.train.get_checkpoint_state(save_dir)
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
return model.sample(sess, words, vocab, n, prime, sample, pick, width)
@app.route("/<int:n>/<int:pick>/<int:width>/<prime>")
def hello(n, pick, width, prime):
return sample(n, pick, width, prime)
if __name__ == "__main__":
app.run(debug=True, use_reloader=True)
First time in browser(Good):
http://127.0.0.1:5000/200/2/4/shakespeare
text:
&C: BIANCA: Believe me, boy; and me, sir, of my troth, I am not so. LADY ANNE: I will not be a man to shrink from him, Or else the duke is great; but to be back'd with the Tower. KING EDWARD IV: Why, here's a man of fourscore three, And make Clowder with the Tower. QUEEN ELIZABETH: Why, then I have not wrong'd. KING RICHARD III: Now, sir, thou liest; and I am loath to see it. Wherein, my lord. KING RICHARD III: Now, sir, thou art not fish; if thou couldst, thou hast not wrong'd. KING RICHARD III: Now, sir, I know you not? KING RICHARD III: Why, let me go: I am in estimation; it is not so far officious; for I am undone! I am too senseless--obstinate, my lord, And give me all. KING RICHARD III: Ay, if you faint, as chaste, by that. BAPTISTA: Where is the matter? DUKE OF YORK: I will not come to me that I am able to see the writing. DUKE OF AUMERLE: My lord, I am a man that mutinies in his own blood, Or in the midst of this bright-shining day, I will not spell. But come, I...
but, the second time :
http://127.0.0.1:5000/200/2/4/shakespeare
I get an error:
ValueError: Variable rnnlm/softmax_w already exists, disallowed. Did you mean to set reuse=True in VarScope? Originally defined at:
File "/home/firestarter/word-rnn-tensorflow-master/model.py", line 54, in __init__
softmax_w = tf.get_variable("softmax_w", [args.rnn_size, args.vocab_size])
File "api.py", line 22, in sample
model = Model(saved_args, True)
File "api.py", line 33, in hello
return sample(n, pick, width, prime)
So from your sample output, the format looks very good, I mean did the model format this for you or you manually separated the paragraphs?
LEONTES:
Why, my Irish time?
And argue in the lord; the man mad, must be deserved a spirit as drown the warlike Pray him, how seven in.
KING would be made that, methoughts I may married a Lord dishonour
Than thou that be mine kites and sinew for his honour
In reason prettily the sudden night upon all shalt bid him thus again. times than one from mine unaccustom'd sir.
LARTIUS:
O,'tis aediles, fight!
Farewell, it himself have saw.
Becuase mine was like this(without any structure)
LEONTES:Why, my Irish time?And argue in the lord; the man mad, must be deserved a spirit as drown the warlike Pray him, how seven in.KING would be made that, methoughts I may married a Lord dishonourThan thou that be mine kites and sinew for his honour
Hey,
I am unable to understand the basis for weighted sampling to pick the next word.
def weighted_pick(weights): t = np.cumsum(weights) s = np.sum(weights) return(int(np.searchsorted(t, np.random.rand(1)*s)))
I also tried with np.argmax(p)
buts its only giving same words predicted all the time.
I first trained the seq to seq model using a UTF-8 encoding (instead of the default ASCII). When sampling from this model, I get the following error:
UnicodeEncodeError: 'ascii' codec can't encode character '\u2018' in position 58: ordinal not in range(128)
How can I fix this?
I'm assuming the amount of time that each batch takes to run shouldn't continuously increase? This is great work by the way!
So I am trying to implemented this RNN word generator model in jupytor notebook. When I was trying to use the trained model to generate some words:
with open(os.path.join(cfgs['save_dir'], 'config.pkl'), 'rb') as f:
saved_args = cPickle.load(f)
with open(os.path.join(cfgs['save_dir'], 'words_vocab.pkl'), 'rb') as f:
words, vocab = cPickle.load(f)
with tf.Session() as sess:
model = Model(saved_args, True)
tf.global_variables_initializer().run()
saver = tf.train.Saver(tf.global_variables())
ckpt = tf.train.get_checkpoint_state(cfgs['save_dir'])
if ckpt and ckpt.model_checkpoint_path:
saver.restore(sess, ckpt.model_checkpoint_path)
print(model.sample(sess, words, vocab, cfgs['n'], cfgs['prime'], cfgs['sample'], cfgs['pick'], cfgs['width']))
It works for the first time, but if I run the code again there is an error:
ValueError: Variable rnnlm/softmax_w already exists, disallowed. Did you mean to set reuse=True in VarScope?
Right now I have to shut down the ipynb file then run the code to get a new sample. Any idea to change the code to avoid this situation?
Hello everyone, I'm new to tensorflow...
The code works perfectly on my machine, but I wonder how to modify
legacy_seq2seq.rnn_decoder()
-> legacy_seq2seq.attention_decoder()
?
I have tried a lot, but it still doesn't work...
So if anyone can tell me how to define attention_states
.
Thanks!
Hi,
I tried this implementation of word-rnn and I have an issue with the train script:
Traceback (most recent call last):
File "train.py", line 111, in
main()
File "train.py", line 48, in main
train(args)
File "train.py", line 93, in train
state = model.initial_state.eval()
AttributeError: 'tuple' object has no attribute 'eval'
I'm using the last version of tensorflow and Python 3.4.
I've already had this issue with one other tensorflow project so I guesss there must be something wrong with my installation.
Thanks
I am a undergraduate research assistant at Purdue University. I am attempting to use the Word RNN you have hosted on github to improve automated monkey testing (by supplying context relevant input instead of random input that the monkey uses). In a few words, I would like to give the RNN a word or phrase (the context) and would like the output of the RNN (for the monkey to use as input) to be the next predicted word. However, right now, the Word RNN implementation looks like it does only a sampling and cannot take input (a phrase or word) and output the next predicted word. Any suggestions on how to go about changing the implementation for my needs?
Example:
In a movie app, we record the following sequence:
⟨Movie,Search,Star Trek\n⟩.
And we train the RNN on similar datasets with these kind of sequences.
Later,
If I give the RNN "Movie, Search" as input, I am looking for the predicted next word, which should be relevant input for the app.
I want to be able to pause training.
Do you have any idea what would be the best way to leverage multiple GPUs?
Hi,
Would you please tell me how to calculate the perplexity using this RNN ? may be some fields or location you can help and fields from where i can get
Cannot reproduce it anymore, but for the record:
hunkim:~/word-rnn-tensorflow$ python sample.py --prime="kmalloc"
kmalloc
Traceback (most recent call last):
File "sample.py", line 42, in <module>
main()
File "sample.py", line 25, in main
sample(args)
File "sample.py", line 39, in sample
print(model.sample(sess, words, vocab, args.n, args.prime, args.sample))
File "/home/hunkim/word-rnn-tensorflow/model.py", line 97, in sample
pred = words[sample]
IndexError: list index out of range
When I use Korean test as the input.txt, error occurs like the following.
UnicodeDecodeError: 'cp949' codec can't decode byte 0xff in position 0: illegal multibyte sequence
how can I make computer learn Korean text?
Plz, help me.
With the new changes training now starts slowing down. First it's normal speed but after about 1000 or so batches it more than doubles the time and eventually it takes almost 50 times as much time to complete a batch and seems to go up geometrically at some point.
My batch times went from 0.7 seconds each to 30 seconds each in under 10,000 batches.
I have read that this usually means the graph has to be finalized or the like but I'm still trying to figure out how that applies here so maybe someone else will beat me to it.
If I stop the training and resume, the time will go back to around 0.7 seconds per batch.
Edit: just removed the graph code and tested and something is still doing it. I reverted to the original code and it doesn't appear to be having the same problems so it's definitely not the graph but there's something causing it for sure.
Since the Windows cmd has a limit of around 5000 words, it is problematic if you want to do a sample greater than that. I modified the sample.py file in my fork so that it append the strings to a file called "output.txt" instead. Feel free to look at it and implement it in your code. I never did python before this, so it may have some issues that I am not personally seeing.
Why do we calculate word counts if we ultimately include all words and sort the vocab alphabetically? Mostly I'm wondering if you ordered it with the most common words first, and later reverted to alphabetical ordering for some reason. Would it be bad to have the most common words first? Or do we expect the model would learn better with a random ordering?
In utils.py:
vocabulary_inv = [x[0] for x in word_counts.most_common()]
vocabulary_inv = list(sorted(vocabulary_inv))
I looked up the documentation for sequence_loss_by_example and it doesn't seem to be taking vocab_size as argument. I'd really appreciate it if you could help me understand what this argument is doing. Thanks a lot!
loss = seq2seq.sequence_loss_by_example([self.logits], [tf.reshape(self.targets, [-1])], [tf.ones([args.batch_size * args.seq_length])], args.vocab_size)
Hello! My server was rebooted and i need to continue train, but have an error:
`[root@adm word-rnn-tensorflow]# python3.6 train.py --data_dir="./data/adm_seotexts" --save_dir="./save/adm_seotexts" --init-from=./save/adm_seotexts
usage: train.py [-h] [--data_dir DATA_DIR] [--input_encoding INPUT_ENCODING]
[--log_dir LOG_DIR] [--save_dir SAVE_DIR]
[--rnn_size RNN_SIZE] [--num_layers NUM_LAYERS]
[--model MODEL] [--batch_size BATCH_SIZE]
[--seq_length SEQ_LENGTH] [--num_epochs NUM_EPOCHS]
[--save_every SAVE_EVERY] [--grad_clip GRAD_CLIP]
[--learning_rate LEARNING_RATE] [--decay_rate DECAY_RATE]
[--gpu_mem GPU_MEM] [--init_from INIT_FROM]
train.py: error: unrecognized arguments: --init-from=./save/adm_seotext`
But I have needed data in save path:
[root@adm word-rnn-tensorflow]# ls save/adm_seotexts/ checkpoint model.ckpt-82000.meta model.ckpt-84000.data-00000-of-00001 model.ckpt-85000.index model.ckpt-86000.meta config.pkl model.ckpt-83000.data-00000-of-00001 model.ckpt-84000.index model.ckpt-85000.meta words_vocab.pkl model.ckpt-82000.data-00000-of-00001 model.ckpt-83000.index model.ckpt-84000.meta model.ckpt-86000.data-00000-of-00001 model.ckpt-82000.index model.ckpt-83000.meta model.ckpt-85000.data-00000-of-00001 model.ckpt-86000.index
Other variants also not working (only --init-from for example):
`[root@tubeadmin word-rnn-tensorflow]# python3.6 train.py --init-from="./save/tubeadmin_seotexts"
usage: train.py [-h] [--data_dir DATA_DIR] [--input_encoding INPUT_ENCODING]
[--log_dir LOG_DIR] [--save_dir SAVE_DIR]
[--rnn_size RNN_SIZE] [--num_layers NUM_LAYERS]
[--model MODEL] [--batch_size BATCH_SIZE]
[--seq_length SEQ_LENGTH] [--num_epochs NUM_EPOCHS]
[--save_every SAVE_EVERY] [--grad_clip GRAD_CLIP]
[--learning_rate LEARNING_RATE] [--decay_rate DECAY_RATE]
[--gpu_mem GPU_MEM] [--init_from INIT_FROM]
train.py: error: unrecognized arguments: --init-from=./save/tubeadmin_seotexts`
How can I run train.py for continue training? Please help) Training ended after 2 weeks, do not want to start from the beginning :)))
Hi
Isn't it possible to retrain the model with new data, to improve output?
When I train the model with my own txt file, then retrain with a new txt file, using the init_from arg, the first issue is the new data and loaded model will obviously disagree on word set and dictionary mappings, but even if these assertions are removed, I still get:
`2018-03-27 10:41:11.721945: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Traceback (most recent call last):
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
return fn(*args)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1302, in _run_fn
status, run_metadata)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
[[Node: save/Assign_26 = Assign[T=DT_FLOAT, _class=["loc:@rnnlm/softmax_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rnnlm/softmax_w/Adam_1, save/RestoreV2_26)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "retrain.py", line 128, in <module>
main()
File "retrain.py", line 56, in main
train(args)
File "retrain.py", line 96, in train
saver.restore(sess, ckpt.model_checkpoint_path)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1666, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 889, in run
run_metadata_ptr)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1120, in _run
feed_dict_tensor, options, run_metadata)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
options, run_metadata)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
[[Node: save/Assign_26 = Assign[T=DT_FLOAT, _class=["loc:@rnnlm/softmax_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rnnlm/softmax_w/Adam_1, save/RestoreV2_26)]]
Caused by op 'save/Assign_26', defined at:
File "retrain.py", line 128, in <module>
main()
File "retrain.py", line 56, in main
train(args)
File "retrain.py", line 93, in train
saver = tf.train.Saver(tf.global_variables())
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1218, in __init__
self.build()
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1227, in build
self._build(self._filename, build_save=True, build_restore=True)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 1263, in _build
build_save=build_save, build_restore=build_restore)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
restore_sequentially, reshape)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 439, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/training/saver.py", line 160, in restore
self.op.get_shape().is_fully_defined())
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/state_ops.py", line 276, in assign
validate_shape=validate_shape)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/ops/gen_state_ops.py", line 57, in assign
use_locking=use_locking, name=name)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
op_def=op_def)
File "/Users/ronanquinn/tensorflow/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
[[Node: save/Assign_26 = Assign[T=DT_FLOAT, _class=["loc:@rnnlm/softmax_w"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rnnlm/softmax_w/Adam_1, save/RestoreV2_26)]]`
Incompatible shapes
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [256,725] rhs shape= [256,726]
How is the init_from arg supposed to be used if not for retraining with new data?
How can i compute blue score on model prediction?
Hello,
Is there a way to give a score for a gine sentence ??? say a perplexity score
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.