Code Monkey home page Code Monkey logo

Comments (12)

crownpku avatar crownpku commented on August 22, 2024 2

非常好的想法!
我还没有仔细看你的代码,先提醒你两个地方:

  1. 确定模型已经load进来了
  2. 确认Flask传进来的字符编码是正确的

也欢迎你提交pull request把这个功能的代码添加进来。

from information-extraction-chinese.

lovenodejs avatar lovenodejs commented on August 22, 2024

你看下我上面的代码,其实问题就是,
with tf.Session(config=tf_config) as sess:
sess.run(tf.global_variables_initializer())
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
print(result)
return json.dumps(result, ensure_ascii=False)

我新加了sess.run(tf.global_variables_initializer()), 这个代码,因为如果不加的话会出现下面的错误:
Traceback (most recent call last):
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "app.py", line 41, in getNameModel
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
File "/home/wzxy/laishaohui/ner/NER_IDCNN_CRF/model.py", line 395, in evaluate_line
trans = self.trans.eval()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 463, in eval
return self._variable.eval(session=session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 606, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3928, in _eval_using_default_session
return session.run(tensors, feed_dict)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value crf_loss/transitions
[[Node: _retval_crf_loss/transitions_0_0 = _RetvalT=DT_FLOAT, index=0, _device="/job:localhost/replica:0/task:0/cpu:0"]]

如果加了sess.run(tf.global_variables_initializer()) 就不会报错,但是模型返回的结果非常差
{"entities": [{"end": 2, "start": 1, "type": "PER", "word": "想"}, {"end": 3, "start": 2, "type": "PER", "word": "集"}, {"end": 7, "start": 6, "type": "LOC", "word": "部"}, {"end": 8, "start": 7, "type": "ORG", "word": "位"}, {"end": 14, "start": 13, "type": "ORG", "word": "席"}, {"end": 15, "start": 12, "type": "LOC", "word": "联团的总于北京,首执"}, {"end": 16, "start": 15, "type": "PER", "word": "行"}], "string": "联想集团的总部位于北京,首席执行官是杨元庆先生"}

就这个问题解决不了,能解决还是非常愿意pull request 的,

from information-extraction-chinese.

lovenodejs avatar lovenodejs commented on August 22, 2024

编码是正确的,模型确实load了
/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory.
"Converting sparse IndexedSlices to a dense Tensor of unknown shape. "
2018-04-12 11:00:14,732 - train.log - INFO - Reading model parameters from ckpt_IDCNN/ner.ckpt
model is loaded

  • Running on http://...:5001/ (Press CTRL+C to quit)
    联想集团的总部位于北京,首席执行官是杨元庆先生

from information-extraction-chinese.

crownpku avatar crownpku commented on August 22, 2024

这是有问题的,你load model时的sess和你做prediction的sess不是同一个session下面,你调用了sess.run(tf.global_variables_initializer())相当于把所有参数都随机化了,模型并没有用到。
需要把整个flask部分放去原先代码中

with tf.Session(config=tf_config) as sess:
    model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)

的同一段代码域里面

from information-extraction-chinese.

lovenodejs avatar lovenodejs commented on August 22, 2024

@app.route('/getNameModel', methods=['POST'])
def predict_line():
config = load_config(FLAGS.config_file)
logger = get_logger(FLAGS.log_file)
# limit GPU memory
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
with open(FLAGS.map_file, "rb") as f:
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
result = ''
with tf.Session(config=tf_config) as sess:
sess.run(tf.global_variables_initializer())
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
title = request.json['title']
print(title)
result = model.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
return json.dumps(result, ensure_ascii=False)
可是这种方式,第一次调用没问题,结果正确,第二次就会报错了

from information-extraction-chinese.

lovenodejs avatar lovenodejs commented on August 22, 2024

每次请求进来都会create_model

from information-extraction-chinese.

crownpku avatar crownpku commented on August 22, 2024
sess = tf.Session(config=tf_config)
sess.run(tf.global_variables_initializer())
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)

然后把sess和model这两个变量一起传给你的flask function试下

from information-extraction-chinese.

lovenodejs avatar lovenodejs commented on August 22, 2024

就像这样:
def predict_line():
config = load_config(FLAGS.config_file)
logger = get_logger(FLAGS.log_file)
# limit GPU memory
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
with open(FLAGS.map_file, "rb") as f:
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
with tf.Session(config=tf_config) as sess:
model = create_model(sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)
return model,sess
返回 model 和 sess

然后在 flask 中接:
predictmodel,sess = predict_line()

print('model is loaded')

@app.route('/getNameModel', methods=['POST'])
def getNameModel():
title = request.json['title']
print(title)
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
with open("maps.pkl", "rb") as f:
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
#print(result)
return json.dumps(result, ensure_ascii=False)

但是会报:

[2018-04-12 16:15:17,870] ERROR in app: Exception on /getNameModel [POST]
Traceback (most recent call last):
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/_compat.py", line 33, in reraise
raise value
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/flask/app.py", line 1598, in dispatch_request
return self.view_functionsrule.endpoint
File "app.py", line 38, in getNameModel
result = predictmodel.evaluate_line(sess, input_from_line(title, char_to_id), id_to_tag)
File "/home/wzxy/laishaohui/ner/NER_IDCNN_CRF/model.py", line 395, in evaluate_line
trans = self.trans.eval()
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/ops/variables.py", line 463, in eval
return self._variable.eval(session=session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 606, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "/root/anaconda2/envs/tensorflow36/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3914, in _eval_using_default_session
raise ValueError("Cannot evaluate tensor using eval(): No default "
ValueError: Cannot evaluate tensor using eval(): No default session is registered. Use with sess.as_default() or pass an explicit session to eval(session=sess)

from information-extraction-chinese.

xtuyaowu avatar xtuyaowu commented on August 22, 2024

这个问题解决了没?

from information-extraction-chinese.

lovenodejs avatar lovenodejs commented on August 22, 2024

还没,很奇怪,总是有问题

from information-extraction-chinese.

xtuyaowu avatar xtuyaowu commented on August 22, 2024

我也试了,没有解决。

from information-extraction-chinese.

tjuGaoxp avatar tjuGaoxp commented on August 22, 2024

这样好像可以:
1、新建py
from flask import Flask,request,json
from main import *

app = Flask(name)

config = load_config(FLAGS.config_file)
logger = get_logger(FLAGS.log_file)

limit GPU memory

tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True
f = open(FLAGS.map_file, "rb")
tf_sess = tf.Session(config=tf_config)
char_to_id, id_to_char, tag_to_id, id_to_tag = pickle.load(f)
model = create_model(tf_sess, Model, FLAGS.ckpt_path, load_word2vec, config, id_to_char, logger)

@app.route('/predict_ner', methods=['POST'])
def predict_ner():
sentence = request.form.to_dict().get('sentence')
print(sentence)
result = model.evaluate_line(tf_sess, input_from_line(sentence, char_to_id), id_to_tag)
print(result)
return json.dumps(result, ensure_ascii=False)

if name == 'main':
app.run(debug='true')

2、修改 model.py 第 395行
trans = self.trans.eval(sess)

from information-extraction-chinese.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.