Comments (9)
顶顶~感觉是写错了吧
from firefly.
写错了启动脚本,更新了readme,应该是下面的脚本,目前建议只使用单卡:
CUDA_VISIBLE_DEVICES=0 python train_qlora.py --train_args_file train_args/qlora.json
from firefly.
写错了启动脚本,更新了readme,应该是下面的脚本,目前建议只使用单卡:
CUDA_VISIBLE_DEVICES=0 python train_qlora.py --train_args_file train_args/qlora.json
我看见qlora里讨论说最新的accelerate已经解决这个问题了,up主快点更新呀! 因为我自己写的代码用多卡会报错Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:7! ,但是我的device_map是auto的,不知道是哪里出了问题呢
from firefly.
目前我也在修改这个bug,看了一些acclerate和qlora的官方实现和issue,把各种库都更新到了最新版本,在4*V100上运行还是会报错,还需要再看一下:
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1645, in train
return inner_training_loop(
File "/opt/conda/lib/python3.8/site-packages/transformers/trainer.py", line 1756, in _inner_training_loop
model, self.optimizer = self.accelerator.prepare(self.model, self.optimizer)
File "/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py", line 1182, in prepare
result = tuple(
File "/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py", line 1183, in <genexpr>
self._prepare_one(obj, first_pass=True, device_placement=d) for obj, d in zip(args, device_placement)
File "/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py", line 1022, in _prepare_one
return self.prepare_model(obj, device_placement=device_placement)
File "/opt/conda/lib/python3.8/site-packages/accelerate/accelerator.py", line 1258, in prepare_model
raise ValueError(
ValueError: You can't train a model that has been loaded in 8-bit precision on a different device than the one you're training on. Make sure you loaded the model on the correct device using for example `device_map={'':torch.cuda.current_device()}you're training on. Make sure you loaded the model on the correct device using for example `device_map={'':torch.cuda.current_device() or device_map={'':torch.xpu.current_device()}
from firefly.
写错了启动脚本,更新了readme,应该是下面的脚本,目前建议只使用单卡:
CUDA_VISIBLE_DEVICES=0 python train_qlora.py --train_args_file train_args/qlora.json
你好 方便建一个交流群吗
from firefly.
目前项目已经支持4-bit多卡并行训练
from firefly.
写错了启动脚本,更新了readme,应该是下面的脚本,目前建议只使用单卡:
CUDA_VISIBLE_DEVICES=0 python train_qlora.py --train_args_file train_args/qlora.json你好 方便建一个交流群吗
感谢关注和支持,我们后续会考虑建立交流群
from firefly.
from firefly.
@yangjianxin1 这个实现多卡量化的关键代码是哪里啊?能展示一下吗?
from firefly.
Related Issues (20)
- 多轮对话训练效果 HOT 1
- deepspeed安装报错 HOT 1
- 训练llama3-8b-it报错 HOT 2
- kto训练实现
- NotImplementedError: No operator found for `memory_efficient_attention_forward`
- RuntimeError: FlashAttention only support fp16 and bf16 data type
- 启动chat.py脚本加载merge_lora生成的checkpoint模型或者adapter参数都会报错
- DPO支持多卡的QLora训练吗?
- unsloth不支持多机多卡,而且对比的还是没有开启flashattn2的增益,有些许疑问
- “max_seq_length:训练时的最大长度。按照自己的设备进行设置,越长需要占用越多显存。”这个有换算规则吗?
- NameError: name 'FastLanguageModel' is not defined
- 使用Unsloth反而OOM了是为什么呢? HOT 2
- 为什么llama2的template中没有bos_token
- Qwen2-7b lora
- ValueError: You can't train a model that has been loaded in 8-bit precision on a different device than the one you're training on. HOT 1
- 请问支持codestral微调和部署吗?
- Qwen2-7B-Instruct 训练loss 0 或推理 probability tensor contains either `inf`, `nan` or element < 0 HOT 6
- qwen2-7b lora loss为0 HOT 1
- expected string or bytes-like object
- 可以支持 llava 微调吗?
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from firefly.