limcheekin / open-text-embeddings Goto Github PK

Looks like the hugging face space no longer exist. https://limcheekin-bge-small-en-v1-5.hf.space/v1

Receiving the following error

Your space is on error, check its status on hf.co

Many open source projects support the compatibility of the completions and the chat/completions endpoints of the OpenAI API, but do not support the embeddings endpoint

I use llama-cpp-python for completions, but wish it supported these embedding-specific models. Now this project comes along and gives me that part! If this project does not intend to deliver completions, would you consider enabling a passthrough backend to serve the completions?

That would allow me to run this project in the front and have it pass through completion requests to llama-cpp-python and finally give me a complete OpenAI compatible solution for both embeddings AND completions without inferior completions from models not purposed for embeddings or changing based on the flavor-of-the-week model I have loaded for completions.

Is it possible? my CUDA reported out of memory.

I have download bge model, how can I load the model from local?

When using then open source models like - jinaai/jina-embeddings-v2-base-en, it throws an error for setting the trust_remote_code to True

Please update the code for handling the same using the below option as mentioned in langchain-ai/langchain#6080

if "jina" in model_name:
        tokenizer = AutoModel.from_pretrained(model_name, trust_remote_code=True)

(vllm-custom) root@llamatest:/vllm/open-text-embeddings# MODEL=intfloat/e5-large-v2 VERBOSE=1 python -m open.text.embeddings.server
Using device: cuda
Loading model: intfloat/e5-large-v2
Normalize embeddings: True
Traceback (most recent call last):
File "/root/anaconda3/envs/vllm-custom/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/root/anaconda3/envs/vllm-custom/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/root/vllm/open-text-embeddings/open/text/embeddings/server/main.py", line 22, in
app = create_app()
File "/root/vllm/open-text-embeddings/open/text/embeddings/server/app.py", line 27, in create_app
initialize_embeddings()
File "/root/vllm/open-text-embeddings/open/text/embeddings/server/app.py", line 121, in initialize_embeddings
embeddings = HuggingFaceInstructEmbeddings(model_name=model_name,
File "/root/anaconda3/envs/vllm-custom/lib/python3.9/site-packages/langchain_community/embeddings/huggingface.py", line 153, in init
self.client = INSTRUCTOR(
File "/root/anaconda3/envs/vllm-custom/lib/python3.9/site-packages/sentence_transformers/SentenceTransformer.py", line 190, in init
modules = self._load_sbert_model(
TypeError: _load_sbert_model() got an unexpected keyword argument 'token'
(vllm-custom) root@llamatest:/vllm/open-text-embeddings# pip show open-text-embeddings
Name: open-text-embeddings
Version: 1.0.4
Summary: Open Source Text Embedding Models with OpenAI API-Compatible Endpoint
Home-page: https://github.com/limcheekin/open-text-embeddings
Author: Lim Chee Kin
Author-email: [email protected]
License: MIT
Location: /root/anaconda3/envs/vllm-custom/lib/python3.9/site-packages
Requires: langchain
Required-by:
(vllm-custom) root@llamatest:/vllm/open-text-embeddings# pip show openai
Name: openai
Version: 1.13.3
Summary: The official Python library for the openai API
Home-page:
Author:
Author-email: OpenAI [email protected]
License:
Location: /root/anaconda3/envs/vllm-custom/lib/python3.9/site-packages
Requires: anyio, distro, httpx, pydantic, sniffio, tqdm, typing-extensions
Required-by:
(vllm-custom) root@llamatest:/vllm/open-text-embeddings#

Download script is also not working
(vllm-custom) root@llamatest:/vllm/open-text-embeddings# ./download.sh intfloat/e5-large-v2
mkdir: missing operand
Try 'mkdir --help' for more information.
Updated git hooks.
Git LFS initialized.
Cloning into 'huggingface.co'...
fatal: repository 'https://huggingface.co/' not found
Not in a git repository.
Git LFS initialized.
/root
total 2672
(vllm-custom) root@llamatest:/vllm/open-text-embeddings#

how to handle such embeddings request?

{"input": [[6719, 263, ], "model": "text-embedding-ada-002", "encoding_format": "base64"}

Seems like unicode.

I tried to pass the local bge model address in theMODEL, but it still tried to connect huggingface. How can I use downloaded model to start the server.

Here is the command:
MODEL=/mnt/bge-base-zh-V1.5 python -m open.text.embeddings.server

Here is the Error Information:
Traceback (most recent call last):
File "/mnt/miniconda/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/mnt/miniconda/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/mnt/miniconda/lib/python3.9/site-packages/open/text/embeddings/server/main.py", line 22, in
app = create_app()
File "/mnt/miniconda/lib/python3.9/site-packages/open/text/embeddings/server/app.py", line 27, in create_app
initialize_embeddings()
File "/mnt/miniconda/lib/python3.9/site-packages/open/text/embeddings/server/app.py", line 116, in initialize_embeddings
tokenizer = AutoTokenizer.from_pretrained(model_name)
File "/mnt/miniconda/lib/python3.9/site-packages/transformers/models/auto/tokenization_auto.py", line 716, in from_pretrained
config = AutoConfig.from_pretrained(
File "/mnt/miniconda/lib/python3.9/site-packages/transformers/models/auto/configuration_auto.py", line 1034, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/mnt/miniconda/lib/python3.9/site-packages/transformers/configuration_utils.py", line 620, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "/mnt/miniconda/lib/python3.9/site-packages/transformers/configuration_utils.py", line 675, in _get_config_dict
resolved_config_file = cached_file(
File "/mnt/miniconda/lib/python3.9/site-packages/transformers/utils/hub.py", line 469, in cached_file
raise EnvironmentError(
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like bge-base-zh-V1.5 is not the path to a directory containing a file named config.json.

Hello author,

I am trying to use this repo as an embedding model backend. This error was raised. Can you help me with that?

Must provide an 'engine' or 'deployment_id' parameter to create a <class 'openai.api_resources.embedding.Embedding'>

Thanks,

• Decompress the request payload with gzip algorithm if gzip is found in Content-Encoding request headers.
• Compress the response with gzip algorithm if gzip is found in Accept-Encoding request headers.