Those should have been the same I think.

Hi, it is very nice for you to write such a great tutorial. I got this from the pytorch official tutorial and I think it can help a lot of people. But do you have any time line of posting the solution of the exercises? I find these exercises useful but I am not sure whether my answer is right since I am not in the area of NLP. So it will be extremely nice if you can give a simple but instructive solution.

Thank you!

hello, I try to run BiLSTM_CRF example. I add one example to training_data and get
"""
training_data = [(
"the wall reported apple corporation made money".split(),
"B I O B I O O".split()
), (
"georgia tech is a university in georgia".split(),
"B I O O O O B".split()
), ("China".split(), 'B'.split())
]
"""
And then, when I run
"""
precheck_sent = prepare_sequence(training_data[2][0], word_to_ix)
print model(precheck_sent)
"""
I get an output 3, which means Start_TAG.

And then I try to change the code of function _viterbi_decode
I change
"""
terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]
best_tag_id = argmax(terminal_var)
path_score = terminal_var[0][best_tag_id]
best_path = [best_tag_id]
"""
to
"""
terminal_var = forward_var + self.transitions[self.tag_to_ix[STOP_TAG]]
temp = terminal_var.data
temp[0][self.tag_to_ix[STOP_TAG]] = -10000
temp[0][self.tag_to_ix[START_TAG]] = -10000
temp = torch.autograd.Variable(temp)
best_tag_id = argmax(temp)
path_score = temp[0][best_tag_id]
best_path = [best_tag_id]`
"""
then I get a correct output 1, which means 'B'.

I wonder to know why it produces a completely wrong answer, START_TAG. Thank you!

Hi,

I'm confused with the way you build word_to_ix here: https://github.com/rguthrie3/DeepLearningForNLPInPytorch/blame/master/Deep%20Learning%20for%20Natural%20Language%20Processing%20with%20Pytorch.ipynb#L1448

Is there any reason of why you don't use the vocab or set(raw_text) instead, like you do here: https://github.com/rguthrie3/DeepLearningForNLPInPytorch/blame/master/Deep%20Learning%20for%20Natural%20Language%20Processing%20with%20Pytorch.ipynb#L1298 ??

Hi Robert,

I followed your Pytorch tutorials for NLP and I implemented 2 exercices that you proposed.
The Cbow and the character level enriching of word embeddings of the POS tagger.

I would appreciate if you review/comment my implementation and why not discuss about it.

Here is the repo's link: https://github.com/MokaddemMouna/Pytorch

Thanks.

The very wonderful notebook is in Python 2

It is all about () in print

Can you please convert to python3 and for those who prefer py2
from future import unicode_literals, print_function, division

Regards
Jayanta/Kolkata/India

"Matrices and vectors are special cases of torch.Tensors, where their dimension is 1 and 2 respectively." I think you meant 2 and 1 respectively. Thanks

I have a question regarding your implementation of NGram (and, regardingly, CBOW which I have adapted). According to the code presented you're creating a two-layer perceptron, and the second linear layer is outputting a tensor of dimensions (out_dim, vocab_size), which is okay as long as you're not trying to train the embeddings on a real corpus with vocab size of, say, 200 thousand tokens which clogs CUDA RAM for good. I cannot see how this makes sense to train the embeddings separately for each batch of several texts. Could you be so kind to explain?

I realise that some of the code are breaks pep8. Mind if I fix those? Blame my OCD. Reading the code makes me a little anxious.

I think you should also give the hidden state and cell state otherwise forward method complains:
lstm_out, self.hidden = self.lstm(embeds.view(len(sentence), 1, -1), self.hidden)

Any plan to convert some of the code base (like the LSTM/NER) to allow for minibatch processing?
Currently, all of them take one instance at a time. Any pointers in that regard would be useful as well.

Hi,

We've included your tutorial in official pytorch tutorials here.
it'll be useful for community to have all the awesome tutorials in one place. I also think that html documentation is more approachable than a notebook.

The current page is almost exactly same as your notebook. I've just raised a PR to break the tutorial into a few parts to make each bite-sized. Would you approve of such a reorganisation? Your feedback would be valuable.

You can contact me at [email protected]

Sasank.

http://pytorch.org/tutorials/beginner/deep_learning_nlp_tutorial.html
has 5 nice noteboks all in python3

The one in github is python2 and not in sync

This may create difficulty for many as it did for me
You may have 1 version - perhaps the 5 working ones

The .creator attribute of the Autograd.variable class appears to have been renamed .grad_fn in the newest release of Pytorch. Thanks for the great tutorial.

model = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM)
optimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4)

Kindly fix

Jayanta/Kolkata/India

TypeError Traceback (most recent call last)
in ()
----> 1 model = BiLSTM_CRF(len(word_to_ix), tag_to_ix, EMBEDDING_DIM, HIDDEN_DIM)
2 optimizer = optim.SGD(model.parameters(), lr=0.01, weight_decay=1e-4)

in init(self, vocab_size, tag_to_ix, embedding_dim, hidden_dim)
27
28 self.word_embeds = nn.Embedding(vocab_size, embedding_dim)
---> 29 self.lstm = nn.LSTM(embedding_dim, hidden_dim/2, num_layers=1, bidirectional=True)
30
31 # Maps the output of the LSTM into tag space.

~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py in init(self, *args, **kwargs)
370
371 def init(self, *args, **kwargs):
--> 372 super(LSTM, self).init('LSTM', *args, **kwargs)
373
374

~/anaconda3/lib/python3.6/site-packages/torch/nn/modules/rnn.py in init(self, mode, input_size, hidden_size, num_layers, bias, batch_first, dropout, bidirectional)
37 layer_input_size = input_size if layer == 0 else hidden_size * num_directions
38
---> 39 w_ih = Parameter(torch.Tensor(gate_size, layer_input_size))
40 w_hh = Parameter(torch.Tensor(gate_size, hidden_size))
41 b_ih = Parameter(torch.Tensor(gate_size))

TypeError: torch.FloatTensor constructor received an invalid combination of arguments - got (float, int), but expected one of:

• no arguments
• (int ...)
  didn't match because some of the arguments have invalid types: (float, int)
• (torch.FloatTensor viewed_tensor)
• (torch.Size size)
• (torch.FloatStorage data)
• (Sequence data)

Check predictions before training

precheck_sent = prepare_sequence(training_data[0][0], word_to_ix)

precheck_tags = torch.LongTensor([ tag_to_ix[t] for t in training_data[0][1] ])

print(model(precheck_sent))