[New Feature] Adding finetune() for spaCy to create customized NER models about mindsdb HOT 3 OPEN

@ZoranPandovski Hi Zoran, I'd love to work on this feature as I said in the chat!

Just one thing: this is how I train the NER model:

optimizer = nlp.initialize()
for itn in range(100):
    random.shuffle(train_data)
    for raw_text, entity_offsets in train_data:
        doc = nlp.make_doc(raw_text)
        example = Example.from_dict(doc, {"entities": entity_offsets})
        nlp.update([example], sgd=optimizer)
nlp.to_disk("/output")

And this is how I load:

nlp = spacy.load("/output")

Is it possible to savee the pipeline/model somewhere and load it? If so which directory/path?

I am thinking of doing f-strings paths for each trained custom pipeline/model. I will use pre-existing NER model and fine-tune it without resetting its internal state.

from mindsdb.

Thanks @HahaBill, I've assigned it to you. For storing the pipeline we can use the model storage, check the base interface

from mindsdb.

@ZoranPandovski Right got it! Thanks for the info!

from mindsdb.