robinvanschaik / interpret-flair Goto Github PK

Flair offers multiple types of text classification models.

From transformers-based models via huggingface (currently implemented) to word embeddings.

Restructuring to allow for other types models would be nice.

Hi Robin,

I found an error when I test with the pre-trained sentiment model from Flair. Simply loading with

classifier = TextClassifier.load('sentiment')
flair_model_wrapper = ModelWrapper(classifier)

And the rest is the same as yours. I got this error when I call the function interpret_sentence.

I test and print out the device. It turns out that the variable input_ids in function interpret_sentence is on cpu. My clumsy solution is to add

input_ids = input_ids.to(device)

after line

input_ids = flair_model_wrapper.tokenizer.encode(...)

There might be other internal solutions.

BTW, this work helps a lot!

I was unable to upload the best-model.pt file as it larger than 100MB. I am looking into GitHub LFS.

The readme.md does not show the red / green word importances in the table.

Replace the table with a screenshot.

As target_label only accept single values, making it work on multi class problems is not so simple. I have written a simple looping function to iterate over the correct target labels, but as on Flair 0.8post1, I find that predictions of the interpret_sentence class deviate from the orginal multi class model (i.e. wrong class prediction).

On personal note: I think you have done a really good job in building this solution. It has worked wonders for multi label problems already.

This branch tracks the progress of adding the SHAP explainer to the repo.

Currently running into some bugs.