zjukg / knowpat Goto Github PK

训练后加载模型推理时报错，请问这是什么原因

Traceback (most recent call last):
File "/cpfs01/shared/KnowPAT/inference.py", line 68, in
model = LlamaForCausalLM.from_pretrained(
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3787, in from_pretrained
model.load_adapter(
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/integrations/peft.py", line 193, in load_adapter
adapter_state_dict = load_peft_weights(peft_model_id, token=token, **adapter_kwargs)
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/peft/utils/save_and_load.py", line 244, in load_peft_weights
has_remote_safetensors_file = file_exists(
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/KnowPAT/checkpoint/nq_5k/checkpoint-1800/'. Use repo_type argument if needed.

加载的checkpoint如下：

请问是怎么实现的呢？我在其他地方看到了一篇关于直接检索的文章，但是他没有给出代码

我用的llama2-7b-hf，服务器用的48GB显存的A40，微调数据集只用了5条数据，为啥就爆显存了？lora方法微调llama2-7b不是不需要那么多显存吗？

Meanwhile,the Atom-7B-chat is first fine-tuned on our cloud product docu�ments which is a domain corpus to enable the backbone model with a basic understanding of the domain concepts. We name this model Atom-7B-CP and employ this fine-tuned version as our backbone

我训练完的checkpoint_name如下

base_path结构如下

May I ask how your answers are ranked and scored? Is it just 4, 3, 2, 1?

您好，我想请问，我在运行inference.py文件时报了这个错误：huggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.
请问 checkpoint_name=YOUR SAVE PATH是一个什么样的路径，base_path是LLAMA的路径吗？ 为什么这里lora_weights = "".format(checkpoint_name)格式化字符串前面是一个空串呀？
能否指点一下，谢谢啦！

你好我这边显示内存不够，请问运行您的程序需要多少内存呀

May I ask if it is possible to publicly disclose the construction code for the knowledge preference set (KPS) and style preference set (SPS) datasets?

Traceback (most recent call last):
File "train.py", line 357, in
train()
File "train.py", line 353, in train
trainer.train()
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/trainer.py", line 1885, in train
return inner_training_loop(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/trainer.py", line 2216, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/trainer.py", line 3216, in training_step
loss = self.compute_loss(model, inputs)
File "train.py", line 288, in compute_loss
logits = model(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/utils/operations.py", line 822, in forward
return model_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/utils/operations.py", line 810, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/peft/peft_model.py", line 1395, in forward
return self.base_model(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/peft/tuners/tuners_utils.py", line 179, in forward
return self.model.forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1205, in forward
outputs = self.model(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1000, in forward
layer_outputs = decoder_layer(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 749, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 351, in forward
query_states = self.q_proj(hidden_states)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/peft/tuners/lora/layer.py", line 540, in forward
result = result + lora_B(lora_A(dropout(x))) * scaling
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/dropout.py", line 59, in forward
return F.dropout(input, self.p, self.training, self.inplace)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/functional.py", line 1252, in dropout
return VF.dropout(input, p, training) if inplace else _VF.dropout(input, p, training)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB (GPU 0; 23.65 GiB total capacity; 22.31 GiB already allocated; 28.56 MiB free; 22.37 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

hello,i have questions above when I use 2 4090.I'd like to ask for the device requirement.Looking forward to your reply.
您好，可以问一下以上问题的解决方案吗？可以提供一些帮助吗？