Code Monkey home page Code Monkey logo

chatbotcourse's Introduction

ChatBotCourse

读本人更多原创文章,欢迎关注微信订阅号

lcsays

欢迎关注我的另外几个github项目

智能游戏AI从基础到实战教程

自己动手做聊天机器人教程

chatbotcourse's People

Contributors

lcdevelop avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

chatbotcourse's Issues

出现Variable embedding_attention_seq2seq/rnn/embedding_wrapper/embedding already exists, disallowed. Did you mean to set reuse=True in VarScope?

ValueError Traceback (most recent call last)
in ()
140 with tf.Session() as sess:
141 sample_encoder_inputs, sample_decoder_inputs ,sample_target_weights= get_samples() #被投喂的数据
--> 142 encoder_inputs, decoder_inputs, target_weights, outputs, loss = get_model() #申请placeholder,前馈得到outputs
143
144 input_feed = {} #投喂数据的键值对,将真实数据投喂到placeholder构成的list--encoder_inputs中,同理对decoder_inputs

in get_model()
131 embedding_size=size,
132 feed_previous=False,
--> 133 dtype=tf.float32)
134
135 #计算交叉熵损失

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\contrib\legacy_seq2seq\python\ops\seq2seq.py in embedding_attention_seq2seq(encoder_inputs, decoder_inputs, cell, num_encoder_symbols, num_decoder_symbols, embedding_size, num_heads, output_projection, feed_previous, dtype, scope, initial_state_attention)
852 embedding_size=embedding_size)
853 encoder_outputs, encoder_state = rnn.static_rnn(
--> 854 encoder_cell, encoder_inputs, dtype=dtype)
855
856 # First calculate a concatenation of encoder outputs to put attention on.

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py in static_rnn(cell, inputs, initial_state, dtype, sequence_length, scope)
1210 state_size=cell.state_size)
1211 else:
-> 1212 (output, state) = call_cell()
1213
1214 outputs.append(output)

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\rnn.py in ()
1197 varscope.reuse_variables()
1198 # pylint: disable=cell-var-from-loop
-> 1199 call_cell = lambda: cell(input_, state)
1200 # pylint: enable=cell-var-from-loop
1201 if sequence_length is not None:

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\rnn_cell_impl.py in call(self, inputs, state, scope)
178 with vs.variable_scope(vs.get_variable_scope(),
179 custom_getter=self._rnn_get_variable):
--> 180 return super(RNNCell, self).call(inputs, state)
181
182 def _rnn_get_variable(self, getter, *args, **kwargs):

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\layers\base.py in call(self, inputs, *args, **kwargs)
439 # Check input assumptions set after layer building, e.g. input shape.
440 self._assert_input_compatibility(inputs)
--> 441 outputs = self.call(inputs, *args, **kwargs)
442
443 # Apply activity regularization.

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\contrib\rnn\python\ops\core_rnn_cell.py in call(self, inputs, state)
112 "embedding", [self._embedding_classes, self._embedding_size],
113 initializer=initializer,
--> 114 dtype=data_type)
115 embedded = embedding_ops.embedding_lookup(embedding,
116 array_ops.reshape(inputs, [-1]))

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\variable_scope.py in get_variable(name, shape, dtype, initializer, regularizer, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)
1063 collections=collections, caching_device=caching_device,
1064 partitioner=partitioner, validate_shape=validate_shape,
-> 1065 use_resource=use_resource, custom_getter=custom_getter)
1066 get_variable_or_local_docstring = (
1067 """%s

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\variable_scope.py in get_variable(self, var_store, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)
960 collections=collections, caching_device=caching_device,
961 partitioner=partitioner, validate_shape=validate_shape,
--> 962 use_resource=use_resource, custom_getter=custom_getter)
963
964 def _get_partitioned_variable(self,

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\variable_scope.py in get_variable(self, name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource, custom_getter)
358 reuse=reuse, trainable=trainable, collections=collections,
359 caching_device=caching_device, partitioner=partitioner,
--> 360 validate_shape=validate_shape, use_resource=use_resource)
361 else:
362 return _true_getter(

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\rnn_cell_impl.py in _rnn_get_variable(self, getter, *args, **kwargs)
181
182 def _rnn_get_variable(self, getter, *args, **kwargs):
--> 183 variable = getter(*args, **kwargs)
184 trainable = (variable in tf_variables.trainable_variables() or
185 (isinstance(variable, tf_variables.PartitionedVariable) and

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\variable_scope.py in _true_getter(name, shape, dtype, initializer, regularizer, reuse, trainable, collections, caching_device, partitioner, validate_shape, use_resource)
350 trainable=trainable, collections=collections,
351 caching_device=caching_device, validate_shape=validate_shape,
--> 352 use_resource=use_resource)
353
354 if custom_getter is not None:

C:\Users\lenovo\Anaconda3\envs\py35\lib\site-packages\tensorflow\python\ops\variable_scope.py in _get_single_variable(self, name, shape, dtype, initializer, regularizer, partition_info, reuse, trainable, collections, caching_device, validate_shape, use_resource)
662 " Did you mean to set reuse=True in VarScope? "
663 "Originally defined at:\n\n%s" % (
--> 664 name, "".join(traceback.format_list(tb))))
665 found_var = self._vars[name]
666 if not shape.is_compatible_with(found_var.get_shape()):

File "demo.py", line 189

(Tensorflow) C:\Users\uxb\Desktop\ChatBotCourse-master\ChatBotCourse-master\chat
botv5>python demo.py train
File "demo.py", line 189
print 'step=', step, 'loss=', loss_ret, 'learning_rate=', learning_rate.eval
()
^
SyntaxError: invalid syntax

(Tensorflow) C:\Users\uxb\Desktop\ChatBotCourse-master\ChatBotCourse-master\chat
botv5>python demo.py predict
File "demo.py", line 189
print 'step=', step, 'loss=', loss_ret, 'learning_rate=', learning_rate.eval
()
^
SyntaxError: invalid syntax

subtile bug ?

ChatBotCourse-master\subtitle\subtitle_crawler\spiders$ python subtitle_spider.py
File "subtitle_spider.py", line 9, in
from subtitle_crawler.items import SubtitleCrawlerItem
ImportError: No module named subtitle_crawler.items

Thank you!

代码请教

[array([0, 0, 0, 0], dtype=int32),
array([0, 0, 0, 0], dtype=int32),
array([8, 3, 5, 3], dtype=int32),
array([7, 8, 2, 1], dtype=int32),
array([6, 2, 10, 9], dtype=int32)]
其中的5个array,表示一句话的长度是5个词,而id,一般通过统计得出,一个id表示一个词,这个array和id到底是什么意思?我的理解,一个array应该代表一句话,每一个id都是句子里的词,但是为什么代码处理时array按列取数值作为encoder_inputs?

chatbotv5..default dictionary是哪一个

loading to token id....
Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 0.477 seconds.
Prefix dict has been built succesfully.
done.....................
load the train data

想问下,default dictionary是哪一个?怎么加载进来的?可以更换吗?

用自己的训练预料loss降不下来

我把训练预料换成了自己准备的,大概10-40w条对话
但是跑完10000个step之后,发现loss在4-5之间。。。
有没有什么办法
谢谢

提示找不到legacy_seq2seq.python.ops

f the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.
"This module will be removed in 0.20.", DeprecationWarning)
Traceback (most recent call last):
File "demo.py", line 7, in
from tensorflow.contrib.legacy_seq2seq.python.ops import seq2seq
ImportError: No module named legacy_seq2seq.python.ops
�大哥,你的tensorflow版本号是多少?

效果不理想,是要更新词库吗?

Loading model cost 1.286 seconds.
Prefix dict has been built succesfully.
2017-12-27 14:20:24.445937: I C:\tf_jenkins\home\workspace\rel-win\M\windows\PY
35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instruct
ions that this TensorFlow binary was not compiled to use: AVX AVX2

hello
WARN:词汇不在服务区
你好
WARN:词汇不在服务区
呵呵

哈哈


WARN:词汇不在服务区

File name too long

Traceback (most recent call last):
File "/usr/lib64/python2.7/site-packages/twisted/internet/defer.py", line 653, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/home/yuliao/subtitle/subtitle_crawler/pipelines.py", line 13, in process_item
fp = open('result/'+file_name, 'w')
IOError: [Errno 36] File name too long: 'result/htt

词嵌入版本

这个chatbot是one-hot做的啊?什么时候更新词嵌入的啊?

???

楼主啊,你写的程序加载vectors就能加载好几个小时啊,别说训练了。
你这个for循环真头痛啊......

^CTraceback (most recent call last):
  File "model.py", line 97, in <module>
    my.train()
  File "model.py", line 81, in train
    trainXY, trainY = self.generate_trainig_data()
  File "model.py", line 17, in generate_trainig_data
    self.vector_dict = vector.load_vectors("vectors.bin") #在此处中断
  File "/data00/home/liyufei.ltitan/python3/vector.py", line 54, in load_vectors
    c = input_file.read(1)
KeyboardInterrupt
######时间
real	31m25.843s
user	23m44.974s
sys	7m33.714s

Python 版本问题

我下载了你的代码并尝试进行编译,出现了一些问题。请问项目中 Python 的版本是 2.x 还是 3.x,或两者都有?

是不是少提交了文件?

python demo.py predict

Building prefix dict from the default dictionary ...
Loading model from cache /tmp/jieba.cache
Loading model cost 0.207 seconds.
Prefix dict has been built succesfully.
2019-01-14 19:45:31.871135: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
WARNING:tensorflow:From demo.py:136: __init__ (from tensorflow.python.ops.rnn_cell_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This class is deprecated, please use tf.nn.rnn_cell.LSTMCell, which supports all the feature this cell currently has. Please replace the existing code with tf.nn.rnn_cell.LSTMCell(name='basic_lstm_cell').
Traceback (most recent call last):
  File "demo.py", line 244, in <module>
    predict()
  File "demo.py", line 205, in predict
    saver.restore(sess, './model/demo')
  File "/usr/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1536, in restore
    if not checkpoint_management.checkpoint_exists(compat.as_text(save_path)):
  File "/usr/lib/python2.7/site-packages/tensorflow/python/training/checkpoint_management.py", line 364, in checkpoint_exists
    if file_io.get_matching_files(pathname):
  File "/usr/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py", line 344, in get_matching_files
    compat.as_bytes(single_filename), status)
  File "/usr/lib/python2.7/site-packages/tensorflow/python/framework/errors_impl.py", line 528, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: ./model; No such file or directory

代码段:
image

代码位置:

# pwd
/home/pzqu/ChatBotCourse/ChatBotCourse/chatbotv5
# ls
demo.py  readme.md  samples  word_token.py  word_token.pyc

数据集

请问能否提供那个21w语料的数据集呢

windows机器

在windows机器,运行chatbot5的demo train 为什么不行,程序直接结束了。

训练无法后台运行

请问让训练后台运行,试了常用几种方法如python demo.py train &都不行。用daemon包强行转守护进程的话一开始即结束

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.