maximumentropy / seq2seq-pytorch Goto Github PK

Hi , MaximumEntropy.
I run your code on pyTorch==0.4 cuda==9.0 and it have many deprecate warning.
could your tell me your pyTorch version .

Hey @MaximumEntropy, thanks for such a nice, clean repo. I was wondering if there was a specific script you used to download the wmt data. Maybe you can point us to what you used?

Also, do you mind sharing how many training examples there are in the WMT data? It looks like you have ~5hr train time per epoch. I was wondering how many training examples was in each epoch.

Hi. In the beam_search.py you have a function get_best(). Shouldn't this return the first element, as in index 0, of the sorted list instead of index 1?

Hey, thanks for your implementations.

Trying to get the summarization code running, but I have a feeling it doesn't actually work (yet). Am i correct to assume so? For example, it's calling read_nmt_data instead of read_summarization_data and you've removed the file in your refactor branch.

Any tips on getting it to run?

Hi, the code has a nice abstraction and easy to follow. Thanks!!

However, there are some issues in your implement....

(code)

If you don't pass c_t through a Linear layer from encoder hidden to decoder hidden, then the code crashes. (Encoder and Decoder can have different dimensions)

(code)

When self.decoder.num_layers != 1 the view function will crash because of dimension dis-match.

Hello!

I am reading your implementation line by line, and found it's nice and easy to follow. Thanks a lot! But I still have some questions. Since I didn't finish reading yet, I guess I will have more later on.

You set the requires_grad of two initial hidden states as false (code). Could you explain why you did this, since I thought they should be true for back-propagation. Also, it is wrong if we set them as true.

Thanks for putting this code up! It's very clean.

My question is similar to #6.

At https://github.com/MaximumEntropy/Seq2Seq-PyTorch/blob/master/nmt.py#L184, it seems like you aren't masking your losses, so outputting pad tokens is part of the supervision. Is this the case?

firstly,thanks for your code,it's really helpful to me,but could i know where is the teacher forcing part,thanks again^_^

@MaximumEntropy Could you, or anyone reading, this share the tokenized version of the data here? It's really important that I run this, but I can't install the Mosesdecoder on my server (that is shared).

While running your code, I encountered this error.

Traceback (most recent call last):
  File "nmt.py", line 181, in <module>
    decoder_logit = model(input_lines_src, input_lines_trg)
  File "/home/cmaurya1/code/py2.7/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/cmaurya1/code/seq2seq/seq2seq_maximum_entropy/model.py", line 841, in forward
    ctx_mask
  File "/home/cmaurya1/code/py2.7/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*in
Traceback (most recent call last):
  File "nmt.py", line 181, in <module>
    decoder_logit = model(input_lines_src, input_lines_trg)
  File "/home/cmaurya1/code/py2.7/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/cmaurya1/code/seq2seq/seq2seq_maximum_entropy/model.py", line 841, in forward
    ctx_mask
  File "/home/cmaurya1/code/py2.7/local/lib/python2.7/site-packages/torch/nn/modules/module.py", line 491, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/cmaurya1/code/seq2seq/seq2seq_maximum_entropy/model.py", line 382, in forward
    output.append(isinstance(hidden, tuple) and hidden[0] or hidden)
RuntimeError: bool value of Tensor with more than one value is ambiguous

Any hint to solve?

In their paper, the input at next time step is the concatenation of h_tilde and the actual input. However, this code seems to use h_tilde as the hidden state for the next time-step LSTM computation.

Brilliant code base, btw.

Hi, your code has a nice abstraction, thanks for your share. But I have a question about the 'attentionLSTM', it seems that you didn' t use any 'ctx_mask' or 'trg_mask' in your code related to attention part, won't this cause error using for attention ? I'm new to pytorch, hope for your reply!

Can not find the data

File "/home/mb75502/Seq2Seq-PyTorch/data_utils.py", line 27, in read_config
json_object = json.load(open(file_path, 'r'))
File "/home/mb75502/anaconda2/lib/python2.7/json/init.py", line 291, in load
**kw)
File "/home/mb75502/anaconda2/lib/python2.7/json/init.py", line 339, in loads
return _default_decoder.decode(s)
File "/home/mb75502/anaconda2/lib/python2.7/json/decoder.py", line 364, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/home/mb75502/anaconda2/lib/python2.7/json/decoder.py", line 380, in raw_decode
obj, end = self.scan_once(s, idx)
ValueError: Expecting property name: line 6 column 3 (char 83)

I use data_utils.py to prosess "fr-en.tgz" data,but have many errors. Could you give a sample data, thanks!

How can I get the attention matrix from the Seq2SeqFastAttention class? I want to put that matrix for visualisation.

Dear authors,

Thanks for sharing you code. Your code is well structured and easy to read, but I still encounter a problem in running seq2seq with beam search.

In the evaluate.py, you have declared that the beam search is in TODO. And in the decode.py I found that the BeamSearchDecoder has been implemented, so I try to run the decode.py, but it throws a Exception like this:

Traceback (most recent call last):
  File "decode.py", line 479, in <module>
    decoder.translate()
  File "decode.py", line 242, in translate
    hypotheses, scores = decoder.decode_batch(j)
  File "decode.py", line 153, in decode_batch
    context
  File "/usr/share/Anaconda2/lib/python2.7/site-packages/torch/nn/modules/module.py", line 206, in __call__
    result = self.forward(*input, **kwargs)
TypeError: forward() takes at most 3 arguments (4 given)

So, could you tell me how to run the decode.py or give me some suggestions for implementing the beam search part. It will be highly appreciated for your any suggestions.

Thanks.