sanxing-chen / nmt2017-zh-en Goto Github PK
View Code? Open in Web Editor NEWPre-processing and training scripts for WMT 2017 ZH-EN translation task
Pre-processing and training scripts for WMT 2017 ZH-EN translation task
2 NVIDA GeForce RTX 2080 Ti
pytorch 1.5.0
include "News Commentary v12" and "UN Parallel Corpus V1.0"
CUDA_VISIBLE_DEVICES=0,1 fairseq-train data-bin/wmt17_en_zh -a transformer --optimizer adam -s en -t zh --label-smoothing 0.1 --dropout 0.3 --max-tokens 4000 --min-lr '1e-09' --lr-scheduler inverse_sqrt --weight-decay 0.0001 --criterion label_smoothed_cross_entropy --max-update 1000000 --warmup-updates 10000 --warmup-init-lr '1e-7' --lr '0.001' --adam-betas '(0.9, 0.98)' --adam-eps '1e-09' --clip-norm 25.0 --keep-last-epochs 10 --save-dir checkpoints_test |& tee -a wmt17_train.test.log
2020-07-28 11:22:01 | INFO | fairseq_cli.generate | Generate test with beam=5: BLEU4 = 0.00, 6.4/0.0/0.0/0.0 (BP=0.444, ratio=0.552, syslen=26013, reflen=47155)
Can you give me some suggestion about this result ?
Thank you !
Hi:
Just want to know How to replicate the result you mentioned on README, The model reaches 20 BLEU on testing dataset, after training for only 2 epochs
.
I simple used your setup to train my model, however after 3 epochs, I got
020-06-03 17:49:03 | INFO | fairseq_cli.generate | Generate test with beam = 5: BLEU4 = 0.09, 7.5/0.7/0.0/0.0 (BP=1.000, ratio=1.996, syslen=289332, reflen=144951)
my generate-script is
fairseq-generate data-bin/wmt17_zh_en \
--path checkpoints/checkpoint_best.pt \
--batch-size 128 --beam 5 --remove-bpe
and the training data I used are:
Thank you!
Since we need to download 'all dataset required in the WMT 2017 MT task'
However, the ftp server inside this cwmt-wmt is broken.
Anyone has these files and are willing to share?
Thanks a lot.
Hi Sanxing, thank you for sharing this script!
I run your preprocess.py
(clean empty lines; I did not run the whole prepare.sh
) and then use fast_align
to learn an alignment model on the parallel corpus.
I found that the perplexity of alignmens given by the alignment model is higher than the results of the parallel corpus preprocessed by another script wmt.py.
I guess this is due to that they merge the blank lines.
So could you possibly add this merge blank lines
function into your script in the future? Thanks a lot!
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.