servereless-runpod-ggml's People
Forkers
tanevanwifferen bitnom example-git bmwas welnity 0x7d0 sockeye44 halusstore l16by heririta git-teoservereless-runpod-ggml's Issues
Stop downloading model at 22%
So I am using https://huggingface.co/TheBloke/OpenAssistant-SFT-7-Llama-30B-GGML , When i do the request and go to container logs i see how it downloads the model until it reach 22%, when it exit me from container logs but it doesn't turn of the worker.
Please rename to serverless-runpod-ggml so it can be found on search
Does it support the GPTQ model?
Hey!
Does it support the GPTQ model? Or will it be supported in the future?
Stream is empty when `COMPLETED` if already read once
Reproduce
- Start a run:
![image](https://private-user-images.githubusercontent.com/1885333/251956542-ac7bb8c3-d902-4b67-a8f8-3ab0d0cf69e7.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxMjMzODgsIm5iZiI6MTcxOTEyMzA4OCwicGF0aCI6Ii8xODg1MzMzLzI1MTk1NjU0Mi1hYzdiYjhjMy1kOTAyLTRiNjctYThmOC0zYWIwZDBjZjY5ZTcucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYyMyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MjNUMDYxMTI4WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9M2JlY2JkYzNmZDJkZTg4MTliYmIyYTQ5ZDMxNTlhNTEzMzhkOGQ5YjhjZDliN2YzZDBmMzI5MTA4MjEyZTFkNSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.3eVtoJ2rz9Ztf7bwoBPDYlTLiNKnyyEFFpe4MJaBNdY)
- Now in progress:
![image](https://private-user-images.githubusercontent.com/1885333/251956312-ae5d3756-eda5-4653-bd98-4de45003e9a3.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxMjMzODgsIm5iZiI6MTcxOTEyMzA4OCwicGF0aCI6Ii8xODg1MzMzLzI1MTk1NjMxMi1hZTVkMzc1Ni1lZGE1LTQ2NTMtYmQ5OC00ZGU0NTAwM2U5YTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYyMyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MjNUMDYxMTI4WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9OTgzZjkyY2RmOThjOGZkY2Y5NGE4NDcxYTIzY2FlOTVjYjgzNmZkMzE2YzU2OTE1Mjc4ZDJhMzQ3NDc0OWI4MiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.3PyFEt3k5lAFznoFYLGBcGLceJa-P2misHDQnc02XKY)
- Wait a while and eventually after checking a couple times I get a
COMPLETED
response with completedstream.output
✅
![image](https://private-user-images.githubusercontent.com/1885333/251956491-344d9956-a1cf-4664-bc4e-3c6a8744812a.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxMjMzODgsIm5iZiI6MTcxOTEyMzA4OCwicGF0aCI6Ii8xODg1MzMzLzI1MTk1NjQ5MS0zNDRkOTk1Ni1hMWNmLTQ2NjQtYmM0ZS0zYzZhODc0NDgxMmEucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYyMyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MjNUMDYxMTI4WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9NmIzMDUyNjQ1YWE1ZGI1ZjNmNmMzOThlZWVkYjlkYTc5N2IzYzk1NGY3MDAyYzU4M2Q2MzhiNDZlYmEwMDY1YSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.DzX-W0McB08lOl4kG__K_IeWLLx3ZwSshURId1vKK4Y)
- Then I check again and it still shows
COMPLETED
except withoutstream.output
❌
![image](https://private-user-images.githubusercontent.com/1885333/251956090-6f5e69e8-325a-485d-92e4-cf5d390bafc8.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTkxMjMzODgsIm5iZiI6MTcxOTEyMzA4OCwicGF0aCI6Ii8xODg1MzMzLzI1MTk1NjA5MC02ZjVlNjllOC0zMjVhLTQ4NWQtOTJlNC1jZjVkMzkwYmFmYzgucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDYyMyUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA2MjNUMDYxMTI4WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MTdmYzJkOTliODhjMWEzMjNlMmMxNTU1ZTk5ZjZmYjk4ZmEzYWZlYmVmZWE3MmY1OGM4ZjM3NmE2NjI4YmQ5ZiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.CSUecKs3opYtp8ki3ST7dRrZXwAAEtEOSBAD_-9qLpc)
Expected: COMPLETED
responses always have stream.output
Why? I'm used to Replicate where you can always get the response given the same job/run ID. Maybe this is incorrect for Runpod, although seems unintuitive since if it deleted the info about the run it shouldn't know it was once COMPLETED
either.
Falcon models support
This worker support falcon models?¿
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.