FileNotFoundError: [Errno 2] No such file or directory: 'SQuAD/meta.msgpack' about drqa HOT 6 CLOSED

run python prepro.py before running python train.py. Please read the instructions carefully.

from drqa.

I ran python prepro.py already and it took over 10 hours to complete but I didn't see the file. If it comes after running the script, I'll try again and see if that helps.

from drqa.

What's your spacy version? if you follow all instructions and install spacy==1.9 the prepro script should only take 1~2 minutes.

from drqa.

@hitvoice
It seems like the annotate function from SpaCy will eat up lots of memory (>10GB)
during the data preparation process. If the machine doesn't have enough RAM, the process will turn out to be extremely slow. I guess it is what @YuanTW was experiencing.

Not sure if it is a memory leakage problem for SpaCy.

from drqa.

@tim5go I didn't pay attention to the memory usage before. I'll check that.

from drqa.

@tim5go I ran the script again and indeed, the memory keeps increasing to about 10G. I didn't figure out the exact reason and I'll add a warning to ReadMe. Thanks for reporting this!

from drqa.