Optional TF-IDF weighting about mycroft HOT 3 CLOSED

Use the TfidfTransformer. Serialize the trained transformer as part of the model.

from mycroft.

Actually does this even make sense? Keras' weighting mechanism only applies to training samples, not at test time. Wouldn't scaling a word vector just turn it into another word? Have to see if there's precedent in the literature for this.

It is troubling that in the experimental results in #20 a bag-of-words SVM does as well as a neural LSTM, because that indicates that I'm not able to effectively make use of sequential data. It's possible that the issue is that I'm treating all words as equally informative in the neural network models. TF-IDF weighting is one way to address this. Would max pooling in a convent model (#17) be another?

from mycroft.

Nah. Not going to do this. Too many other avenues to explore.

from mycroft.