Couldn't instantiate the backend tokenizer from one of: (1) a `tokenizers` library serialization file, (2) a slow tokenizer instance to convert or (3) an equivalent slow tokenizer class to instantiate and convert. You need to have sentencepiece installed to convert a slow tokenizer to a fast one. about question_generation HOT 4 CLOSED

@Xyde
Can you tall me what happen and how to solve it? I get same probelm now

I slove it to pip install transformers==3.0.0

from question_generation.

@Xyde
Can you tall me what happen and how to solve it? I get same probelm now

from question_generation.

Yes. I had same problem and I found that I used transformers 4.2. The solution was just same as @nicken described.

It will be helpful if we get an update code for transformers 4.2. Is there a plan for this?

from question_generation.

I got a error while evaluate valhalla/t5-base-qg-hl model using 'nlp = pipeline("question-generation", model="valhalla/t5-base-qg-hl", ans_model='valhalla/t5-base-qa-qg-hl', qg_format="highlight")'.
'AssertionError: You should supply an encoding or a list of encodings to this method. An encoding is the output of one the encoding methods of the tokenizer, i.e. call/encode_plus/batch_encode_plus.'

Any help would be appreciated.

from question_generation.