jungokasai / graph_parser Goto Github PK
View Code? Open in Web Editor NEWSOTA TAG Parser
SOTA TAG Parser
Hi Jungo,
I'm interested in your work and I'm trying to run your pre-trained model. I followed the procedure as in your readme file, downloaded the GLOVE and PRE-TRAINED model in corresponding directory. However, it gives me this error. It seems like the problem(so far) is the inconsistent shape of word embedding layer.
Full information listed as follow:
Run joint training. Use gold supertags
D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained/Pretrained_Parser
Found 39549 unique lowercased words including -unseen- and <-root->.
Indexing word vectors.
Found 400000 word vectors.
Found 400000 words not in the training set but in the glove data
end glove indexing
Found 80 unique characters including -unseen-. NOT including <-root->.
Found 47 unique POS tags including -unseen- and <-root->.
D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\utils\data_loader\data_process_secsplit.py:159: FutureWarning: arrays to stack must be passed as a "sequence" type suc
h as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.
self.gold_jk = np.hstack(map(lambda x: x[1:], jk_sequences[self.nb_train_samples:]))
Found 4728 unique supertags including -unseen- and <-root->.
D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\utils\data_loader\data_process_secsplit.py:187: FutureWarning: arrays to stack must be passed as a "sequence" type suc
h as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.
self.gold_stags = np.hstack(map(lambda x: x[1:], tag_sequences[self.nb_train_samples:]))
Found 11 unique rels including -unseen-, NOT including <-root->.
D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\utils\data_loader\data_process_secsplit.py:214: FutureWarning: arrays to stack must be passed as a "sequence" type suc
h as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.
self.gold_rels = np.hstack(map(lambda x: x[1:], rel_sequences[self.nb_train_samples:]))
2019-01-18 17:21:22.497084: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 1334, in _do_call
return fn(*args)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 1319, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 1407, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [439550,100] rhs shape= [405687,100]
[[{{node save/Assign_443}} = Assign[T=DT_FLOAT, _class=["loc:@word_embedding/word_embedding_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task
:0/device:CPU:0"](word_embedding/word_embedding_mat/Adam, save/RestoreV2:443)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1546, in restore
{self.saver_def.filename_tensor_name: save_path})
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 929, in run
run_metadata_ptr)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 1152, in _run
feed_dict_tensor, options, run_metadata)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 1328, in _do_run
run_metadata)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\client\session.py", line 1348, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [439550,100] rhs shape= [405687,100]
[[node save/Assign_443 (defined at D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py:60) = Assign[T=DT_FLOAT, _class=["loc:@word_embeddi
ng/word_embedding_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](word_embedding/word_embedding_mat/Adam, save/RestoreV2:443)]]
Caused by op 'save/Assign_443', defined at:
File "D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/graph_parser_main.py", line 134, in
run_model_test(options, opts)
File "D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py", line 60, in run_model_test
saver = tf.train.Saver(max_to_keep=1)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1102, in init
self.build()
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1114, in build
self._build(self._filename, build_save=True, build_restore=True)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1151, in _build
build_save=build_save, build_restore=build_restore)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 795, in _build_internal
restore_sequentially, reshape)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 428, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 119, in restore
self.op.get_shape().is_fully_defined())
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\ops\state_ops.py", line 221, in assign
validate_shape=validate_shape)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 64, in assign
use_locking=use_locking, name=name)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\framework\ops.py", line 3274, in create_op
op_def=op_def)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [439550,100] rhs shape= [405687,100]
[[node save/Assign_443 (defined at D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py:60) = Assign[T=DT_FLOAT, _class=["loc:@word_embeddi
ng/word_embedding_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](word_embedding/word_embedding_mat/Adam, save/RestoreV2:443)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/graph_parser_main.py", line 134, in
run_model_test(options, opts)
File "D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py", line 63, in run_model_test
saver.restore(session, test_opts.modelname)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1582, in restore
err, "a mismatch between the current graph and the graph")
tensorflow.python.framework.errors_impl.InvalidArgumentError: Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the ch
eckpoint. Please ensure that you have not altered the graph expected based on the checkpoint. Original error:
Assign requires shapes of both tensors to match. lhs shape= [439550,100] rhs shape= [405687,100]
[[node save/Assign_443 (defined at D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py:60) = Assign[T=DT_FLOAT, _class=["loc:@word_embeddi
ng/word_embedding_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](word_embedding/word_embedding_mat/Adam, save/RestoreV2:443)]]
Caused by op 'save/Assign_443', defined at:
File "D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/graph_parser_main.py", line 134, in
run_model_test(options, opts)
File "D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py", line 60, in run_model_test
saver = tf.train.Saver(max_to_keep=1)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1102, in init
self.build()
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1114, in build
self._build(self._filename, build_save=True, build_restore=True)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 1151, in _build
build_save=build_save, build_restore=build_restore)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 795, in _build_internal
restore_sequentially, reshape)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 428, in _AddRestoreOps
assign_ops.append(saveable.restore(saveable_tensors, shapes))
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\training\saver.py", line 119, in restore
self.op.get_shape().is_fully_defined())
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\ops\state_ops.py", line 221, in assign
validate_shape=validate_shape)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\ops\gen_state_ops.py", line 64, in assign
use_locking=use_locking, name=name)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\util\deprecation.py", line 488, in new_func
return func(*args, **kwargs)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\framework\ops.py", line 3274, in create_op
op_def=op_def)
File "D:\research\Anaconda3\envs\tf_cpu\lib\site-packages\tensorflow\python\framework\ops.py", line 1770, in init
self._traceback = tf_stack.extract_stack()
InvalidArgumentError (see above for traceback): Restoring from checkpoint failed. This is most likely due to a mismatch between the current graph and the graph from the checkpoint. Plea
se ensure that you have not altered the graph expected based on the checkpoint. Original error:
Assign requires shapes of both tensors to match. lhs shape= [439550,100] rhs shape= [405687,100]
[[node save/Assign_443 (defined at D:\Research\PyCharmProject\graph_parser-master\graph_parser-master\graph_parser_model.py:60) = Assign[T=DT_FLOAT, _class=["loc:@word_embeddi
ng/word_embedding_mat"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](word_embedding/word_embedding_mat/Adam, save/RestoreV2:443)]]
Traceback (most recent call last):
File "D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/scripts/run_pretrained.py", line 117, in
nmsl_parser(config_file, best_model, data_types, opts.no_gold, opts.pretrained)
File "D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/scripts/run_pretrained.py", line 105, in nmsl_parser
subprocess.check_call(complete_command, shell=True)
File "D:\Research\Anaconda3\envs\tf_cpu\lib\subprocess.py", line 271, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'D:/research/Anaconda3/envs/tf_cpu/python.exe D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/graph_parser_main.py test --base_
dir D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained --pretrained --pretrained --model D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pr
etrained/Pretrained_Parser/best_model --predicted_stags_file D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\predicted_stag\test.txt --predicted_pos_file D
:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\predicted_pos\test.txt --predicted_arcs_file D:/Research/PyCharmProject/graph_parser-master/graph_parser-mas
ter/pretrained\predicted_arcs\test.txt --predicted_rels_file D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\predicted_rels\test.txt --predicted_arcs_file_
greedy D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\predicted_arcs_greedy\test.txt --predicted_rels_file_greedy D:/Research/PyCharmProject/graph_parser-
master/graph_parser-master/pretrained\predicted_rels_greedy\test.txt --text_test D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\sents\test.txt --jk_test D
:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\sents\test.txt --tag_test D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\sent
s\test.txt --arc_test D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\sents\test.txt --rel_test D:/Research/PyCharmProject/graph_parser-master/graph_parser
-master/pretrained\sents\test.txt --punc_test D:/Research/PyCharmProject/graph_parser-master/graph_parser-master/pretrained\punc\test.txt --metrics NoPunct_LAS_Both NoPunct_LAS NoPunct_
UAS Stagging POS' returned non-zero exit status 1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.