Comments (6)
embeddings support has been merged to master. It is experimental and currently it's available only on llama.cpp
based models, so any feedback is more than welcome!
To enable it you can set embeddings: true
in the model's YAML config file
from localai.
I've published a sample using embeddings over here: https://github.com/go-skynet/LocalAI/tree/master/examples/query_data
from localai.
Just curious to find out what is the use/purpose of embeddings above.
For the following use case of Retrieval Augmented Data QA:
https://blog.langchain.dev/tutorial-chatgpt-over-your-data/
Can't we use the following embedding models? I plan to use gpt4all-j
with one of the following embeddings model.
- https://huggingface.co/sentence-transformers/all-mpnet-base-v2
- https://huggingface.co/flax-sentence-embeddings/all_datasets_v3_mpnet-base
Please advise. Thank you.
from localai.
further optimizations in #222 - now embeddings can be used with bert on any model - and there is also a huge performance impact!
from localai.
Hello! I am trying to run a gpt4all-j model for building a local chatbot. How can I use an embedding using BERT and implement it for chat completions endpoint?
Currently, I am running it on Mac Mini i7, 32gb RAM. I am planning to upgrade it to a higher resource(vRAM) cloud server in future. Is it possible to make a fast chatbot API using own document embeddings?
from localai.
https://github.com/go-skynet/LocalAI/tree/master/examples/query_data
Thank you for the example! But it can't be included in the API? Currently I think you run those commands inside the container, right? Is there already the scenario that calling a certain path executes the query on the documents?
from localai.
Related Issues (20)
- Feature Discussion: Role-Based Auth HOT 2
- The API v1/images/generations do not apply the size
- Please integrate chatTTS HOT 1
- Error grabbing logs: invalid character '\x00' looking for beginning of value
- Distributed Llama.cpp Inferencing option `llamacpp-worker` not working HOT 2
- animagine-xl not working on latest-cpu
- Site missing "WEBUI" info HOT 1
- Chat WebUI stops updating HOT 1
- Rerank API not accessible: {"error":{"code":500,"message":"grpc service not ready","type":""}} HOT 1
- Parler-tts doesn't work when installed from gallery, documentation unhelpful
- coqui tts: change to better maintained fork HOT 2
- Transformers backend supports mps
- API appears to hang forever if a response is interrupted HOT 1
- support models from OCI registry
- Timeout
- add zsh/bash autocompletion to local-ai CLI
- Feature request: Request the ability to upload a private SSL certificate provided by a secure solution for downloading models HOT 1
- Feature request: Request the ability to upload a private SSL certificate provided by a secure solution for downloading models HOT 2
- Quickstart not working HOT 7
- model qwen2-7b-instruct HOT 8
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from localai.