Comments (3)
正常
from llama-factory.
--per_device_train_batch_size 1
from llama-factory.
明白,batch为1时可以降现存。我想问下4*80G,zero2,offload cpu的情况下,batch开到8就会oom,想问下正常吗?
from llama-factory.
Related Issues (20)
- llamafactory-cli train -h 初始化报错 HOT 3
- 老师,metric.py中pred最终出现乱码怎么处理?
- V100 glm4 loss异常 HOT 2
- 关于gen_kwargs["eos_token_id"]设置的问题 HOT 1
- 使用单机多卡微调Qwen2-72B HOT 2
- qlora和lora训练的区别是啥 HOT 4
- 使用 chatglm2 的 template 对chatglm2 进行微调,出现 INFO-Cannot add this chat template to tokenizer HOT 1
- How to supervised fine tuning Qwen2-7b using Llama2 template? HOT 1
- 0.8.1版本DeepSpeed 的 zero stage3报错 HOT 6
- 单卡deepspeed & lora对glm4-9b进行sft微调报错:RuntimeError: 'weight' must be 2-D HOT 1
- 使用qwen7b对训练好的sft权重合并之后,进行chat,出现keyerror错误
- AttributeError: 'PreTrainedTokenizerFast' object has no attribute 'image_processor' HOT 1
- CUDA显存不足 HOT 1
- 推理的时候出现错误:RuntimeError: CUDA error: device-side assert triggered HOT 3
- BAdam能支持多GPU训练吗? HOT 2
- SFT yi-34B 保存断点时候报错 HOT 1
- ValueError: Unrecognized configuration class <class 'transformers.models.llava.configuration_llava.LlavaConfig'> for this kind of AutoModel: AutoModelForCausalLM. HOT 5
- 无法进行推理,可以微调以及加载模型。 HOT 11
- qwen1.5-7b 预训练lora不收敛 HOT 4
- Qwen2 7B SFT 无法启动训练 HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from llama-factory.