Comments (12)
非常好的想法!
我还没有仔细看你的代码,先提醒你两个地方:
- 确定模型已经load进来了
- 确认Flask传进来的字符编码是正确的
也欢迎你提交pull request把这个功能的代码添加进来。
from information-extraction-chinese.
你看下我上面的代码,其实问题就是,
with tf.Session(config=tf_config) as sess:
sess.run(tf.global_variables_initializer())
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
print(result)
return json.dumps(result, ensure_ascii=False)
我新加了sess.run(tf.global_variables_initializer()), 这个代码,因为如果不加的话会出现下面的错误:
Traceback (most recent call last):
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "app.py", line 41, in getNameModel
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
File "/home/wzxy/laishaohui/ner/NER_IDCNN_CRF/model.py", line 395, in evaluate_line
trans = self.trans.eval()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 463, in eval
return self._variable.eval(session=session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 606, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3928, in _eval_using_default_session
return session.run(tensors, feed_dict)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value crf_loss/transitions
[[Node: _retval_crf_loss/transitions_0_0 = _RetvalT=DT_FLOAT, index=0, _device="/job:localhost/replica:0/task:0/cpu:0"]]
如果加了sess.run(tf.global_variables_initializer()) 就不会报错,但是模型返回的结果非常差
{"entities": [{"end": 2, "start": 1, "type": "PER", "word": "想"}, {"end": 3, "start": 2, "type": "PER", "word": "集"}, {"end": 7, "start": 6, "type": "LOC", "word": "部"}, {"end": 8, "start": 7, "type": "ORG", "word": "位"}, {"end": 14, "start": 13, "type": "ORG", "word": "席"}, {"end": 15, "start": 12, "type": "LOC", "word": "联团的总于北京,首执"}, {"end": 16, "start": 15, "type": "PER", "word": "行"}], "string": "联想集团的总部位于北京,首席执行官是杨元庆先生"}
就这个问题解决不了,能解决还是非常愿意pull request 的,
from information-extraction-chinese.
编码是正确的,模型确实load了
/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2018-04-12 11:00:14,732 - train.log - INFO - Reading model parameters from ckpt_IDCNN/ner.ckpt
model is loaded
- Running on http://...:5001/ (Press CTRL+C to quit)
联想集团的总部位于北京,首席执行官是杨元庆先生
from information-extraction-chinese.
这是有问题的,你load model时的sess和你做prediction的sess不是同一个session下面,你调用了sess.run(tf.global_variables_initializer())相当于把所有参数都随机化了,模型并没有用到。
需要把整个flask部分放去原先代码中
with tf.Session(config=tf_config) as sess:
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
的同一段代码域里面
from information-extraction-chinese.
@app.route('/getNameModel', methods=['POST'])
def predict_line():
config = load_config(FLAGS.config_file)
logger = get_logger(FLAGS.log_file)
# limit GPU memory
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
with open(FLAGS.map_file, "rb") as f:
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
result = ''
with tf.Session(config=tf_config) as sess:
sess.run(tf.global_variables_initializer())
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
title = request.json['title']
print(title)
result = model.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
return json.dumps(result, ensure_ascii=False)
可是这种方式,第一次调用没问题,结果正确,第二次就会报错了
from information-extraction-chinese.
每次请求进来都会create_model
from information-extraction-chinese.
sess = tf.Session(config=tf_config)
sess.run(tf.global_variables_initializer())
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
然后把sess和model这两个变量一起传给你的flask function试下
from information-extraction-chinese.
就像这样:
def predict_line():
config = load_config(FLAGS.config_file)
logger = get_logger(FLAGS.log_file)
# limit GPU memory
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
with open(FLAGS.map_file, "rb") as f:
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
with tf.Session(config=tf_config) as sess:
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
return model,sess
返回 model 和 sess
然后在 flask 中接:
predictmodel,sess = predict_line()
print('model is loaded')
@app.route('/getNameModel', methods=['POST'])
def getNameModel():
title = request.json['title']
print(title)
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
with open("maps.pkl", "rb") as f:
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
#print(result)
return json.dumps(result, ensure_ascii=False)
但是会报:
[2018-04-12 16:15:17,870] ERROR in app: Exception on /getNameModel [POST]
Traceback (most recent call last):
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "app.py", line 38, in getNameModel
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
File "/home/wzxy/laishaohui/ner/NER_IDCNN_CRF/model.py", line 395, in evaluate_line
trans = self.trans.eval()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 463, in eval
return self._variable.eval(session=session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 606, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3914, in _eval_using_default_session
raise ValueError("Cannot evaluate tensor using eval()
: No default "
ValueError: Cannot evaluate tensor using eval()
: No default session is registered. Use with sess.as_default()
or pass an explicit session to eval(session=sess)
from information-extraction-chinese.
这个问题解决了没?
from information-extraction-chinese.
还没,很奇怪,总是有问题
from information-extraction-chinese.
我也试了,没有解决。
from information-extraction-chinese.
这样好像可以:
1、新建py
from flask import Flask,request,json
from main import *
app = Flask(name)
config = load_config(FLAGS.config_file)
logger = get_logger(FLAGS.log_file)
limit GPU memory
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
f = open(FLAGS.map_file, "rb")
tf_sess = tf.Session(config=tf_config)
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
model = create_model(tf_sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
@app.route('/predict_ner', methods=['POST'])
def predict_ner():
sentence = request.form.to_dict().get('sentence')
print(sentence)
result = model.evaluate_line(tf_sess, input_from_line(sentence, char_to_id), id_to_tag)
print(result)
return json.dumps(result, ensure_ascii=False)
if name == 'main':
app.run(debug='true')
2、修改 model.py 第 395行
trans = self.trans.eval(sess)
from information-extraction-chinese.
Related Issues (20)
- 请问关于NER_IDCNN_CRF中关于训练次数的疑问
- 请问关于NER_IDCNN_CRF中关于训练次数的疑问
- 关于运行结果不一致的问题 HOT 1
- 关于network和train里的参数问题 HOT 1
- 【seg_features】的长度问题
- 作者你好,想请问能否公布一下如何获取数据的代码
- test_GRU如何计算F1值 HOT 2
- GRU模型如何计算F1,recall,如果想根据关系的类别查看准确率该怎么做呢
- 训练新标签
- 如何识别新加的实体类型
- 数据标注工具
- 如何训练自己的数据呢?数据怎么标注?可不可以把实体标签换成自己设计的呢?
- 单个关系一般需要多少的数据量进行训练可以达到一个理想的效果
- 作者 你好,关于关系抽取那块test_GRU跑出来的precision超级低 HOT 1
- 测试输出报错:InvalidArgumentError (see above for traceback): ConcatOp : Dimensions
- test_GRU.py文件如何输出测试准确率呢? HOT 1
- test_GRU
- test_GRU运行第一个main函数,出现错误
- RE关系抽取中数据集的vec.txt
- 让test生成三元组并保存,该怎么做呢
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from information-extraction-chinese.