zjukg / knowpat Goto Github PK
View Code? Open in Web Editor NEW[Paper][ACL 2024 Findings] Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering
Home Page: https://arxiv.org/abs/2311.06503
[Paper][ACL 2024 Findings] Knowledgeable Preference Alignment for LLMs in Domain-specific Question Answering
Home Page: https://arxiv.org/abs/2311.06503
训练后加载模型推理时报错,请问这是什么原因
Traceback (most recent call last):
File "/cpfs01/shared/KnowPAT/inference.py", line 68, in
model = LlamaForCausalLM.from_pretrained(
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3787, in from_pretrained
model.load_adapter(
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/transformers/integrations/peft.py", line 193, in load_adapter
adapter_state_dict = load_peft_weights(peft_model_id, token=token, **adapter_kwargs)
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/peft/utils/save_and_load.py", line 244, in load_peft_weights
has_remote_safetensors_file = file_exists(
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 110, in _inner_fn
validate_repo_id(arg_value)
File "/opt/conda/envs/llama_factory/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 158, in validate_repo_id
raise HFValidationError(
huggingface_hub.utils._validators.HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': '/KnowPAT/checkpoint/nq_5k/checkpoint-1800/'. Use repo_type
argument if needed.
请问是怎么实现的呢?我在其他地方看到了一篇关于直接检索的文章,但是他没有给出代码
我用的llama2-7b-hf,服务器用的48GB显存的A40,微调数据集只用了5条数据,为啥就爆显存了?lora方法微调llama2-7b不是不需要那么多显存吗?
Meanwhile,the Atom-7B-chat is first fine-tuned on our cloud product docu�ments which is a domain corpus to enable the backbone model with a basic understanding of the domain concepts. We name this model Atom-7B-CP and employ this fine-tuned version as our backbone
May I ask how your answers are ranked and scored? Is it just 4, 3, 2, 1?
您好,我想请问,我在运行inference.py文件时报了这个错误:huggingface_hub.utils.validators.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: ''.
请问 checkpoint_name=YOUR SAVE PATH是一个什么样的路径,base_path是LLAMA的路径吗? 为什么这里lora_weights = "".format(checkpoint_name)格式化字符串前面是一个空串呀?
能否指点一下,谢谢啦!
你好我这边显示内存不够,请问运行您的程序需要多少内存呀
May I ask if it is possible to publicly disclose the construction code for the knowledge preference set (KPS) and style preference set (SPS) datasets?
Traceback (most recent call last):
File "train.py", line 357, in
train()
File "train.py", line 353, in train
trainer.train()
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/trainer.py", line 1885, in train
return inner_training_loop(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/trainer.py", line 2216, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/trainer.py", line 3216, in training_step
loss = self.compute_loss(model, inputs)
File "train.py", line 288, in compute_loss
logits = model(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/utils/operations.py", line 822, in forward
return model_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/utils/operations.py", line 810, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/amp/autocast_mode.py", line 14, in decorate_autocast
return func(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/peft/peft_model.py", line 1395, in forward
return self.base_model(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/peft/tuners/tuners_utils.py", line 179, in forward
return self.model.forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1205, in forward
outputs = self.model(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 1000, in forward
layer_outputs = decoder_layer(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 749, in forward
hidden_states, self_attn_weights, present_key_value = self.self_attn(
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 351, in forward
query_states = self.q_proj(hidden_states)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/peft/tuners/lora/layer.py", line 540, in forward
result = result + lora_B(lora_A(dropout(x))) * scaling
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1190, in _call_impl
return forward_call(*input, **kwargs)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/modules/dropout.py", line 59, in forward
return F.dropout(input, self.p, self.training, self.inplace)
File "/home/ymr/anaconda3/envs/alignment/lib/python3.8/site-packages/torch/nn/functional.py", line 1252, in dropout
return VF.dropout(input, p, training) if inplace else _VF.dropout(input, p, training)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 32.00 MiB (GPU 0; 23.65 GiB total capacity; 22.31 GiB already allocated; 28.56 MiB free; 22.37 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
hello,i have questions above when I use 2 4090.I'd like to ask for the device requirement.Looking forward to your reply.
您好,可以问一下以上问题的解决方案吗?可以提供一些帮助吗?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.