Comments (1)
model=TheBloke_Llama-2-13B-GPTQ/model.safetensors, I also tried: Wizard-Vicuna-7B-Uncensored-GPTQ-4bit-128g.no-act-order.safetensors, same problem
Loading model ... --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) Cell In[36], line 5 1 # MODEL_PATH = '/home/quang/working/LLMs/oobabooga_linux/text-generation-webui/models/TheBloke_Wizard-Vicuna-7B-Uncensored-GPTQ' 2 # CHECKPOINT_PATH = '/home/quang/working/LLMs/oobabooga_linux/text-generation-webui/models/TheBloke_Wizard-Vicuna-7B-Uncensored-GPTQ/Wizard-Vicuna-7B-Uncensored-GPTQ-4bit-128g.no-act-order.safetensors' 4 DEVICE = torch.device('cuda:0') ----> 5 model, tokenizer = load_model_main(MODEL_PATH, CHECKPOINT_PATH, DEVICE) File /mnt/data/generativeAgent_LLM/server/model.py:56, in load_model_main(model_para, checkpoint_para, device) 55 def load_model_main(model_para, checkpoint_para, device): ---> 56 model = load_quant(model_para, checkpoint_para, 4, 128) 57 model.to(device) 58 tokenizer = AutoTokenizer.from_pretrained(model_para) File /mnt/data/generativeAgent_LLM/server/model.py:47, in load_quant(model, checkpoint, wbits, groupsize) 45 if checkpoint.endswith('.safetensors'): 46 from safetensors.torch import load_file as safe_load ---> 47 model.load_state_dict(safe_load(checkpoint)) 48 else: 49 model.load_state_dict(torch.load(checkpoint)) File ~/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py:1671, in Module.load_state_dict(self, state_dict, strict) 1666 error_msgs.insert( 1667 0, 'Missing key(s) in state_dict: {}. '.format( 1668 ', '.join('"{}"'.format(k) for k in missing_keys))) 1670 if len(error_msgs) > 0: -> 1671 raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( 1672 self.__class__.__name__, "\n\t".join(error_msgs))) 1673 return _IncompatibleKeys(missing_keys, unexpected_keys) RuntimeError: Error(s) in loading state_dict for LlamaForCausalLM: Unexpected key(s) in state_dict: "model.layers.0.self_attn.rotary_emb.inv_freq", "model.layers.0.self_attn.k_proj.g_idx", "model.layers.0.self_attn.o_proj.g_idx", "model.layers.0.self_attn.q_proj.g_idx", "model.layers.0.self_attn.v_proj.g_idx", "model.layers.0.mlp.down_proj.g_idx", "model.layers.0.mlp.gate_proj.g_idx", "model.layers.0.mlp.up_proj.g_idx", "model.layers.1.self_attn.rotary_emb.inv_freq", "model.layers.1.self_attn.k_proj.g_idx", "model.layers.1.self_attn.o_proj.g_idx", "model.layers.1.self_attn.q_proj.g_idx", "model.layers.1.self_attn.v_proj.g_idx", "model.layers.1.mlp.down_proj.g_idx", "model.layers.1.mlp.gate_proj.g_idx", "model.layers.1.mlp.up_proj.g_idx", "model.layers.2.self_attn.rotary_emb.inv_freq", "model.layers.2.self_attn.k_proj.g_idx", "model.layers.2.self_attn.o_proj.g_idx", "model.layers.2.self_attn.q_proj.g_idx", "model.layers.2.self_attn.v_proj.g_idx", "model.layers.2.mlp.down_proj.g_idx", "model.layers.2.mlp.gate_proj.g_idx",
@myrainbowandsky @QuangBK I was also getting the same error but it was specific to this model, I later tried with microsoft phi 2 and mixtral and those two worked fine.
from generativeagent_llm.
Related Issues (8)
- SafetensorError: Error while deserializing header: HeaderTooLarge
- How could i run this code with openai api, The OpenAI API does not support Guidance pattern controls. HOT 1
- Why the score add "1-vertor_relevance"
- keyerror items when adding memories HOT 3
- BaseRetriever.get_relevant_documents() takes 2 positional arguments but 3 were given HOT 1
- Loop HOT 1
- Share a requirement.txt HOT 3
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from generativeagent_llm.