Comments (4)
Hi, StarCoder2 is now integrated in llama.cpp: ggerganov/llama.cpp#5795
(from: https://x.com/sourab_m/status/1764583142499954942?s=20)
from starcoder2.
awful random users sweating to make some ugly cross compatibility that you owners of the project should be taking care about and including it in the release.
I have to disagree, things implemented by the community are in many cases much better than what the authors can come up with, hence the great power of open-source ๐
And FYI the llama.cpp integration is perfectly functional and was done by an HF employee, you can use it in Ollama as mentioned in the tweet.
from starcoder2.
Hi there, sorry but I'm not sure how this is related to my request. This is exactly what I am complaining about - awful random users sweating to make some ugly cross compatibility that you owners of the project should be taking care about and including it in the release.
How hard is it to start quantizing your own models as well along the release? C'mon ๐
An official implementation by your team would ensure optimal performance and compatibility, eliminating the need for community-driven workarounds.
from starcoder2.
Can't see where the GGUF quantized models are? The hugging face repo does not seem contain them yet? I suggest reading the initial post again. Is it hard to start quantizing your own models from day zero? Why would you want someone else to do that for you?
Open source is awesome but this time it looks more like some lazy effort from the team, specially for a project like starcoder2 that had some veery nice monetary support from Nvidia, ServiceNow and others.
The links you mentioned are more like a pull request thing with a lot of problem solving involved. For the mass end user this is barely useful.
The project is beautiful, useful and very well crafted. Hope to see more people using it.
Thanks.
from starcoder2.
Related Issues (17)
- What does "unique tokens" mean (in the paper) ? HOT 2
- What prevents you from throughly opensourcing? HOT 4
- Can't load StarCoder2-3B HOT 4
- Some weights of the model checkpoint at `finetune_starcoder2/final_checkpoint were not used when initializing Starcoder2ForCausalLM HOT 4
- what is the sft template๏ผ HOT 1
- format for inference in code completion HOT 6
- Can starcoder2 be trained with a different language like TCL or lisp? HOT 1
- Clash in requirements for finetuning Starcoder2 HOT 3
- Better inference based on starcode2-3b model HOT 1
- support SPM mode for FIM prompts HOT 2
- Megatron model weights for StarCoder2-15B HOT 2
- Unlawful use of my code HOT 2
- "lm_head.weight" not in the parameters of Starcoder2-3B and Starcoder2-7B (Huggingface version) HOT 2
- Inquiry about Fine-Tuning Using Custom Code HOT 1
- CrossCodeEval Results for StarCoder 2
- [Q] Does this support fill-in-the-middle completion?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from starcoder2.