Comments (2)
Huggingface Transformers is explicity supported. In particular, bge
is linked in this very readme:
https://github.com/michaelfeil/infinity#or-launch-the-cli-using-a-pre-built-docker-container
In total, you could switch for bge-small
between three backends: onnx (fastembed
), ctranslate2 (ctranslate2
), and huggingface transformers (torch
) using the --engine
param.
@BBC-Esq Pre-converted models in CTranslate2 are not available. For Encoders/Embeddings, CTranslate2 offers only moderate speedup, I would suggest using torch
from infinity.
I believe they're already supported...Check here, although I haven't this guy's model's specifically, I've noticed that he's very active and so I'm assuming he converts them correctly. If this answers your question, please close this issue in your next response. Thanks.
https://huggingface.co/michaelfeil/ct2fast-e5-large
from infinity.
Related Issues (20)
- infinity_emb failed at startup using `torch.compile` when installed via pip HOT 8
- Reranker model fails to load (maidalun1020/bce-reranker-base_v1) - no max token length is set HOT 4
- "msg":"Input should be a valid list" HOT 6
- Content-Encoding: gzip HOT 7
- Adding mkdocs url
- shrink: docker image size by pruning venv HOT 6
- Question: Support for sparse embeddings? HOT 2
- Multi-Modal Inference / Clip HOT 1
- Update docs based on feeback. HOT 2
- Issue templates HOT 1
- [Docs] Add quantization / dtype doc
- Dynamic loading - different models at request time / multiple models HOT 4
- Move `.detach().cpu()` into `encode_core`, and option to use cuda streams HOT 5
- Love the repo! Wish I could help! HOT 1
- benchmarks? HOT 1
- float16 and other optimizations help? HOT 6
- How to run or access infinity on hf a space? HOT 1
- The embeddings are random When use multithreading requests HOT 5
- Safetensors or to be sure not to load pickled weights HOT 3
- Does this work with re-rankers?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from infinity.