arminkmz / im2latex Goto Github PK

[Trainer.py], 2020-10-24 22:51:56: start training one epoch
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:1614: UserWarning: nn.functional.tanh is deprecated. Use torch.tanh instead.
  warnings.warn("nn.functional.tanh is deprecated. Use torch.tanh instead.")
[Trainer.py], 2020-10-24 22:51:58: Batch 0: loss=4.466525077819824, acc=0.04311377245508982, lr=0.001
[Trainer.py], 2020-10-24 22:52:01: Epoch finished, loss=4.300304094950358 acc=0.1813689656592087, lr=0.001
[Trainer.py], 2020-10-24 22:52:01: training one epoch finished.
[Trainer.py], 2020-10-24 22:52:01: Epoch 1 saved.
/usr/local/lib/python3.6/dist-packages/nltk/translate/bleu_score.py:490: UserWarning: 
Corpus/Sentence contains 0 counts of 2-gram overlaps.
BLEU scores might be undesirable; use SmoothingFunction().
  warnings.warn(_msg)
/usr/local/lib/python3.6/dist-packages/nltk/translate/bleu_score.py:490: UserWarning: 
Corpus/Sentence contains 0 counts of 3-gram overlaps.
BLEU scores might be undesirable; use SmoothingFunction().
  warnings.warn(_msg)
[Training.train], 2020-10-24 19:22:02: BLEU Score: 0.219257
[Training.train], 2020-10-24 19:22:02: Edit Distance Accuracy: 0.023256
[Trainer.py], 2020-10-24 22:52:03: checkpoints/snapshot-01.pt loaded.
[Trainer.py], 2020-10-24 22:52:03: evaluation starts.
[Trainer.py], 2020-10-24 22:52:04: evaluation finished.
[Training.evaluate], 2020-10-24 19:22:04: loss=3.8671720027923584, acc=0.16326530612244897
[Training.evaluate], 2020-10-24 19:22:05: BLEU Score: 0.000188
[Training.evaluate], 2020-10-24 19:22:05: Edit Distance Accuracy: 0.054217
[Trainer.py], 2020-10-24 22:52:05: start training one epoch
Traceback (most recent call last):
  File "main.py", line 171, in <module>
    train(config)
  File "main.py", line 89, in train
    predictions, epoch_loss, epoch_acc = _trainer.train_one_epoch()
  File "/content/im2latex-1/src/trainer.py", line 59, in train_one_epoch
    logits = self.model(x_train, y_train, self.teacher_forcing_ratio)
  File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/content/im2latex-1/src/model.py", line 116, in forward
    next_token = output.multinomial(1).squeeze(1) # (batch)
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

/pytorch/aten/src/ATen/native/cudnn/RNN.cpp:1266: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
/pytorch/aten/src/ATen/native/cudnn/RNN.cpp:1266: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
/pytorch/aten/src/ATen/native/cudnn/RNN.cpp:1266: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
/pytorch/aten/src/ATen/native/cudnn/RNN.cpp:1266: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters().
...