Custom preprocessing in Live Test about pyss3 HOT 8 CLOSED

I've just finished making those changes and released the new version (0.5.9). I've also updated the notebook adding a section for "Hyperparameter Optimization". Try performing hyperparameter optimization similar to what I did in that notebook and let me know. In case you are still getting bad accuracy, please share some more details, like part of the actual code, the actual accuracy before and after the changes, etc. It would be much easier to try to help that way. I hope you achieve great accuracy again 😢 🤞 🍀

from pyss3.

Hi @enthussb!
Sure, I overlooked this option when first coding the Live Test tool. Thank you for your suggestion :)

I've added this feature in the new version, and also took the opportunity to incorporate some other things that were pending, namely, what's new on this version is:

• The Live Test Tool now supports custom (user-defined) preprocessing methods (b50cfaf, resolved #3).
• The tokenization process was improved (26fff88, 4af8e80).
• The process for recognizing word n-grams during classification was improved (2ceb148).

Update your package version using the pip install -U pyss3 to the new version (0.5.8). To make things easier for you, I've created a new Jupyter Notebook in the examples folder in which is shown how to work incorporate user-defined preprocessing functions to the Live Test tool visualizations for you to follow if you want: using_custom_preprocessing.ipynb

Let me know if everything worked OK ☕

from pyss3.

@all-contributors would you add @enthussb for ideas to the README file? it helped to make this project better by suggesting this cool feature 👍

from pyss3.

@sergioburdisso

I've put up a pull request to add @enthussb! 🎉

from pyss3.

@sergioburdisso I updated the package and ran the code. Everything is working fine, although my accuracy has been reduced quite a bit. I guess it might be due to the latest n-gram and tokenization changes. Could you please have a look at that?

from pyss3.

I was about to tell you to perform a hyperparameter optimization using the Evaluation.grid_search() function but then I realized that I didn't include a "prep" argument to disable the default preprocessing. As a consequence, users won't be able to perform any hyperparameter optimization using only their custom preprocessing method. I'll work on that and add the "prep" parameter to the grid_search(), test, and kfold_cross_validation functions of the Evaluation class. I'm sorry for forgetting to add this in the first place 😢. I'll notify you as soon as the new version is released.

from pyss3.

Okay no problem 👍, till then I can work on the previous version where I had achieved great accuracy!

from pyss3.

Sure I will check the updated package and revert.

from pyss3.