Bug: Does any convert.py supprt llama3? about llama.cpp HOT 7 CLOSED

See huggingface/transformers#30334, this is not our script and sadly we can't provide support for it.

from llama.cpp.

You'll want to use convert-hf-to-gguf.py. The Llama3 BPE pretokenizer is supported by default in convert-hf-to-gguf-update.py.

from llama.cpp.

How to convert origin Meta-Llama-3 to gguf?

❯ ls models/Meta-Llama-3-8B
checklist.chk       consolidated.00.pth params.json         tokenizer.model

from llama.cpp.

Convert the .pth weights yourself using https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py, or download the weights from huggingface. Then you can use convert-hf-to-gguf.py

from llama.cpp.

Convert the .pth weights yourself using https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/convert_llama_weights_to_hf.py, or download the weights from huggingface. Then you can use convert-hf-to-gguf.py

I have tried it:

python src/transformers/models/llama/convert_llama_weights_to_hf.py --input_dir ../llama.cpp/models/Meta-Llama-3-8B --output_dir converted

but it produce error:

.venv/lib/python3.10/site-packages/sentencepiece/__init__.py", line 316, in LoadFromFile
    return _sentencepiece.SentencePieceProcessor_LoadFromFile(self, arg)
RuntimeError: Internal: could not parse ModelProto from ../llama.cpp/models/Meta-Llama-3-8B/tokenizer.model

from llama.cpp.

Thanks a lot!

from llama.cpp.

If you update to the latest version of transformers the scripts supports the conversion

from llama.cpp.