Bigram and trigram collection files are not as listed about lexpredict-lexnlp HOT 9 CLOSED

Hi @JonathanHourany and @DomHudson , we've created a hotfix branch and uploaded a tarball with the new files coming in 0.1.9:
https://github.com/LexPredict/lexpredict-lexnlp/blob/0.1.8-hotfix-stopwords-collocations/lexnlp/nlp/en/stopwords_collocations_hotfix_0.1.8.tar.gz

1. You'll note that we've changed the collocation sizes, in addition to fixing the underlying issue with their automation.

2. You'll also note that we now provide multiple stopword files. The proportion in the file name corresponds to the cumulative token frequency represented by the stopwords; e.g., stopwords_0.5.pickle corresponds to the top N frequency words s.t. 50% of all token occurrences are represented by these stopwords.

3. These are still preliminary for public use and we are working on releasing the underlying data generation/models in the near future. For example, you may or may not want numeric or hyphenated tokens in your stopwords. You may have seen our release of OpenEDGAR (https://github.com/LexPredict/openedgar) last week, which will be a dependency for customizing these

from lexpredict-lexnlp.

Thanks, @JonathanHourany . We'll investigate this in vendoring/distribution and add some unit tests to cover going forward.

from lexpredict-lexnlp.

Hello @JonathanHourany , we have identified the issue and have fixes pending for the next release. Would you like us to ship you updated copies in the meantime?

from lexpredict-lexnlp.

@mjbommar Yes, please, that would be great!

from lexpredict-lexnlp.

I am also interested in this if possible!
Many thanks,
Dom

from lexpredict-lexnlp.

@JonathanHourany - Please email us at [email protected] so we can get you the files.

Thanks,
Eric Detterman
[email protected]

from lexpredict-lexnlp.

@DomHudson Please email us at [email protected] so we can get you the files.

Thanks,
Eric Detterman
[email protected]

from lexpredict-lexnlp.

@ericlex - Will do. Thanks!

from lexpredict-lexnlp.

Many thanks!

from lexpredict-lexnlp.