GPU utilization about sw_study HOT 3 CLOSED

Hi, I uploaded a run.sh script for training. Feel free to have a look and try to run it on your machine. :)

Several rules of thumb to accelerate the training speed:

• use large batch size (1024 for my exps);
• use mul-thread script, i.e. mul_sw2vec.py. Depending on your GPU RAM and CPU core numbers, for large corpus, use as many threads as possible.

For example, a single 1080ti with 16 cpu thread machine, I remember 600m en corpus will have 20-30k word/sec speed (3-4 threads). And based on your speed and the number of words to be trained (shown in log file, Words in train file), you should be able to calculate how long it takes to be trained. :)

from sw_study.

I investigated it more, it actually uses the GPU, but it still is very slow.
for such a training corpus and bpe method, how long it would take to be trained?

from sw_study.

Thanks very much, I was using sw2vec.py, and I was confused that why it changes the number of threads to 1. I will try your suggestions and let you know about its outcome.

from sw_study.