Data file structure/creation about sentences HOT 3 CLOSED

Hey!

Thanks for submitting the issue, it's great to see that people are trying to use this library.

The files were actually created from nltk punkt tokenizer training data. At this time this library was meant to be a 1:1 port of the library so I ended up using their training data.

I'll follow up with what the keys actually mean when I can get a chance to review the data structures -- it's been awhile.

If you wanted to use your own training data, we would have to write the nltk trainer, I attempted to do it at one point but found grabbing their data easier.

https://github.com/nltk/nltk/blob/develop/nltk/tokenize/punkt.py

from sentences.

Ah, ok- thank you! I'll probably just use the training file generated by nltk for the moment, in the interests of just getting it done, but I'll take a look at implementing it. Could be a fun problem.

from sentences.

Closing for now, please feel free to ask follow up questions and I'll re-open.

from sentences.