how to pre-process CNN/DM？ about text-summarizer-pytorch HOT 6 CLOSED

I have found this : https://github.com/abisee/cnn-dailymail
Is that what I should do ？

from text-summarizer-pytorch.

Yeah.. Also, cnn-dailymail abstracts are created by merging multiple sentences (multiple python strings) which is not the case of gigaword abstracts (single python string). So you need to take care of this difference while using my batcher.py file. I suggest you to take a look at this where they create .bin files for cnn-dailymail in pytorch

from text-summarizer-pytorch.

thank you very much～

from text-summarizer-pytorch.

@Stephen-Adams Could you please share your modifications?

from text-summarizer-pytorch.

@rohithreddy024 can you explain why use bin file but not txt file? Thanks.

from text-summarizer-pytorch.

@kscp123 I am using the same batcher.py file as that of this and this. So I am following their bin format convention. As for as the use cases of binary formatted input, you can just google them.

from text-summarizer-pytorch.