RuntimeError: CUDA error: out of memory about unsupervisedmt HOT 5 CLOSED

Hello,

Can you have a look at #13 or #14 ? If this doesn't solve your problem, please tell me.

from unsupervisedmt.

Thank you very much. It‘s finally work！And on newstest2014 en-fr, it give above 23.33 BLEU after 25 epochs. But there is another question. When I change my translation task to Mongolian-Chinese，I modified the /NMT/get_data.sh and got some data (/data/mono/all.mn.tok.6000.pth, /data/mono/all.zh.tok.6000.pth, /data/para/dev/val1w.XX.6000.pth, /data/para/dev/test1w.XX.6000.pth, /data/mono/all.mn-zh.6000.vec).

Then I run the command as follows:
python main.py --exp_name mn-zh --transformer True --n_enc_layers 3 --n_dec_layers 3 --share_enc 2 --share_dec 2 --max_len=100 --share_lang_emb True --share_output_emb True --langs 'mn,zh' --n_mono -1 --mono_dataset 'mn:./data/mono/all.mn.tok.6000.pth,,;zh:./data/mono/all.zh.tok.6000.pth,,' --para_dataset 'mn-zh:,./data/para/dev/val1w.XX.6000.pth,./data/para/dev/test1w.XX.6000.pth' --mono_directions 'mn,zh' --word_shuffle 3 --word_dropout 0.1 --word_blank 0.2 --pivo_directions 'zh-mn-zh,mn-zh-mn' --pretrained_emb './data/mono/all.mn-zh.6000.vec' --pretrained_out True --lambda_xe_mono '0:1,100000:0.1,300000:0' --lambda_xe_otfd 1 --otf_num_processes 8 --otf_sync_params_every 1000 --enc_optimizer adam,lr=0.0001 --epoch_size 20 --stopping_criterion bleu_mn_zh_valid,1

But the Bleu is 0, I want to know if NMT can be used for Mongolian-Chinese translation tasks. What should I do?

from unsupervisedmt.

You are only using 6000 BPE codes is that expected? Usually I use 60000. I tried 8000 once and it gave me pretty bad results too. Also, did you evaluate the word translation accuracy of your cross-lingual BPE embeddings? If you had a mn-zh dictionary you could evaluate the word translation accuracy using https://github.com/facebookresearch/MUSE or something.

Also, how big are your monolingual corpora? It looks like the training epoch finished in 8 seconds. You also set --epoch_size 20, why? Usually epoch size is around 100000 or something.

from unsupervisedmt.

The number of monolingual sentences for each language is 1230000，and when I used 60000 BPE codes， CUDA error: out of memory is come again，so I changed it to 6000.

I will try to evaluate the word translation accuracy using https://github.com/facebookresearch/MUSE . Thank you！

from unsupervisedmt.

Hi，I run the unsupervised NMT codes and PBSMT codes，but the BLEU are both 0. Is there too little monolingual corpus?

NMT codes：
python main.py --exp_name mn-zh --transformer True --n_enc_layers 2 --n_dec_layers 2 --share_enc 1 --share_dec 1 --max_len=50 --share_lang_emb True --share_encdec_emb True --share_decpro_emb True --share_output_emb True --langs 'mn,zh' --n_mono -1 --mono_dataset 'mn:./data/mono/all.mn.tok.40000.pth,,;zh:./data/mono/all.zh.tok.40000.pth,,' --para_dataset 'mn-zh:,./data/para/dev/val3k.XX.40000.pth,./data/para/dev/test3k.XX.40000.pth' --mono_directions 'mn,zh' --word_shuffle 3 --word_dropout 0.1 --word_blank 0.2 --pivo_directions 'zh-mn-zh,mn-zh-mn' --pretrained_emb './data/mono/all.mn-zh.40000.vec' --pretrained_out True --lambda_xe_mono '0:1,100000:0.1,300000:0' --lambda_xe_otfd 1 --otf_num_processes 8 --otf_sync_params_every 1000 --enc_optimizer adam,lr=0.0001 --epoch_size 100000 --stopping_criterion bleu_mn_zh_valid,10 --batch_size 32

PBSMT codes：

MUSE codes：

from unsupervisedmt.