Comments (4)
可以,修改MP_SIZE即可:
from cogvlm.
可以,修改MP_SIZE即可:
感谢!所以我单机4卡就是NUM_GPUS_PER_WORKER=4,MP_SIZE=4对吧?(因为line 22的LOCAL_WORLD_SIZE=$NUM_GPUS_PER_WORKER让我不太确定NUM_GPUS_PER_WORKER这个参数的含义了,以为WORLD_SIZE是实际的进程数,会变成4x4=16需要16张卡)
另外微调cogagent大概需要多少显存呢?我4卡似乎还是不够
from cogvlm.
微调lora的话8张3090吧。4卡的话你可以尝试减少一点微调参数,比如只训练language model的lm_head,并且去掉lora,不知道行不行。
from cogvlm.
微调lora的话8张3090吧。4卡的话你可以尝试减少一点微调参数,比如只训练language model的lm_head,并且去掉lora,不知道行不行。
好的,感谢
from cogvlm.
Related Issues (20)
- Chat using one image and three prompt HOT 1
- CogVLM是开放中文模型了吗,开源模型是否已经支持中文提问回答以及中文数据微调呢? HOT 1
- [CogVLM-chat-v1.1] LM weights are different with vicuna-7b-v1.5 HOT 3
- Running Gradio app locally results in inappropriate error: "NETWORK ERROR DUE TO HIGH TRAFFIC. PLEASE REGENERATE OR REFRESH THIS PAGE." HOT 1
- Using CogVLM as an API HOT 1
- Code of finetuning the cogagent on Mind2Web ? HOT 1
- Deploy CogVLM using Docker
- Could we replace the vicuna-7b directly with stronger llm? HOT 1
- 我想用同样的promt,在每次都清除上下文的情况下得到3种答案,为什么结果都是一样的 HOT 2
- Chat with PDF documentation instead of images
- CogAgent 视觉预训练模型 EVA2-CLIP-L HOT 1
- CogVLM源代码是否支持多轮对话训练 HOT 5
- 关于模型视觉定位原理
- 运行微调脚本报错缺少相关参数 HOT 2
- 如何构建CogAgent的微调数据集? HOT 1
- 两张3090微调CogVLM的可能性? HOT 1
- 加载cogvlm-chat-hf模型报错 Error while deserializing header: MetadataIncompleteBuffer
- 我该使用什么格式的输入来用模型进行visual grounding 任务? HOT 1
- 原来带grounding功能的是哪个web demo地址? HOT 1
- Cogagent demo can not be accessed HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from cogvlm.