feature: embedding support about localai HOT 6 CLOSED

embeddings support has been merged to master. It is experimental and currently it's available only on llama.cpp based models, so any feedback is more than welcome!

To enable it you can set embeddings: true in the model's YAML config file

from localai.

I've published a sample using embeddings over here: https://github.com/go-skynet/LocalAI/tree/master/examples/query_data

from localai.

Just curious to find out what is the use/purpose of embeddings above.

For the following use case of Retrieval Augmented Data QA:
https://blog.langchain.dev/tutorial-chatgpt-over-your-data/

Can't we use the following embedding models? I plan to use gpt4all-j with one of the following embeddings model.

• https://huggingface.co/sentence-transformers/all-mpnet-base-v2
• https://huggingface.co/flax-sentence-embeddings/all_datasets_v3_mpnet-base

Please advise. Thank you.

from localai.

further optimizations in #222 - now embeddings can be used with bert on any model - and there is also a huge performance impact!

from localai.

Hello! I am trying to run a gpt4all-j model for building a local chatbot. How can I use an embedding using BERT and implement it for chat completions endpoint?

Currently, I am running it on Mac Mini i7, 32gb RAM. I am planning to upgrade it to a higher resource(vRAM) cloud server in future. Is it possible to make a fast chatbot API using own document embeddings?

from localai.

https://github.com/go-skynet/LocalAI/tree/master/examples/query_data

Thank you for the example! But it can't be included in the API? Currently I think you run those commands inside the container, right? Is there already the scenario that calling a certain path executes the query on the documents?

from localai.