Comments (2)
aarnphm
commented on May 15, 2024
Hey there, you are running a sync function within async context. This is not recommended.
The reason for 528f76e is to use httpx within an async call. This assumes that all function within this context are async function.
llm.run is a sync call, which is blocking. Please use the async variant of Langchain run.
from openllm.
aarnphm
commented on May 15, 2024
This is not a bug, but rather a usage issue.
from openllm.
Related Issues (20)
- bug: cannot load local model by model-id
- bug: Output text from CompletionChunk is different with tokenizer.decode HOT 7
- bug: microsoft/phi-2 hangs on macos i7
- bug: Linux Mint: The Service throws errors when getting model-requests HOT 1
- Docker image for version 0.4.41 not built
- bug: when use backend=pt in normal or adapter mode,RuntimeError: Exception caught during generation: Failed to parse JSON from SSE message:(vllm is ok) HOT 2
- bug: 'torch.dtype' object has no attribute 'lower'
- running out of memory
- bug: Inference fails for `docker run` with `google/flan-t5-large` model with `pytorch` on CPU HOT 1
- bug: block GPU HOT 1
- Add a gradio demo for the python sdk section
- Why change prompt_token_ids depending on encoder_decoder
- bug: RuntimeError: Exception caught during generation: Response payload is not completed
- bug: [WARNING] [api_server:llm-llama-service:3] Timed out waiting for runner to be ready
- Unable to fetch models.
- bug: Not able to work with locally downloaded model
- How to modify the port number?
- How to load a model offline HOT 1
- bug: "Failed to parse JSON from SSE message" error HOT 3
- bug: AttributeError in Phi-2 Model Initialization Using BentoML OpenLLM Docker Container HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from openllm.