Comments (8)
catmeowjiao
commented on July 21, 2024
transformers==4.27.1也可以
from chatglm.cpp.
catmeowjiao
commented on July 21, 2024
我明白了，必须得用量化后的模型是因为我的电脑内存不足，直接加载正常的模型内存不够用，就崩了
from chatglm.cpp.
catmeowjiao
commented on July 21, 2024
chatglm3没有提供量化过的模型，所以内存小于32G且使用CPU的无解
from chatglm.cpp.
catmeowjiao
commented on July 21, 2024
只能用别人量化好的
from chatglm.cpp.
catmeowjiao
commented on July 21, 2024
intel developer cloud终于维护完了，我从那上面量化一下
from chatglm.cpp.
sammichenVV
commented on July 21, 2024
请问下你们有试过GPU版本的么，我试了怎么编译都是用CPU推理
from chatglm.cpp.
TGJek
commented on July 21, 2024
chatglm3没有提供量化过的模型，所以内存小于32G且使用CPU的无解
我这边系统32G的内存，Windows 11操作系统，量化模型的时候因为系统还开着其他的程序所以内存不太够，但是Windows换页换页着，最后还是把模型量化好了。
from chatglm.cpp.
TGJek
commented on July 21, 2024
请问下你们有试过GPU版本的么，我试了怎么编译都是用CPU推理
你可以试一试构建程序前给GGML机器学习库的编译配置加上一个“使用CUBLAS”的选项
from chatglm.cpp.
Related Issues (20)
- CodeGeeX2模型转换错误 HOT 2
- ChatGLM3 使用 examples/cli_demo.py 时报错 'TypeError: chat() got an unexpected keyword argument 'max_new_tokens'' HOT 1
- check failed (data != MAP_FAILED) Invalid argument HOT 1
- How to construct a ToolCallMessage?
- 更新后运行报错：No module named 'pydantic._internal' HOT 4
- 【已解决，附解决方法】无法运行 LangChain API ，执行命令后报错， 能否提供能成功运行此命令的 python版本号 和 使用的环境中安装的包的列表 HOT 3
- 显存使用增加
- 为什么4090显卡用不了，编译成功之后也是用的cpu推理 HOT 5
- mac m1 启动cli_demo.py时，抛出incompatible architecture异常 HOT 1
- 如何设置更长的上下文(context) HOT 5
- v0.3.0 "发布" 里的Python包中的C++程序是以什么配置构建的？
- Fix: Dockerfile Building Locally
- 量化后的ChatGlm3-6B使用open_api部署tools功能无法使用 HOT 1
- -DGGML_CUBLAS=ON，编译成功之后无法使用显卡进行推理，仍然使用CPU进行推理 HOT 4
- Long context result (>450 tokens) from server will stop / return incorrect json (stream mode)
- openai_api 返回为空 HOT 1
- 能否在开启openai api时候提供一个public link？ HOT 1
- Run the following command to report an error python3 chatglm3_demo.py -m ../chatglm3-ggml.bin -p 你好 --temp 0.8 --top_p 0.8
- zsh: illegal hardware instruction HOT 1
- 请问下这个报错是什么原因造成的？ HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from chatglm.cpp.