Model architecture? about yi HOT 5 CLOSED

We adopt llama2 model architecture, but the layernorm's names are different for our experiment purpose.
You can refer to https://huggingface.co/01-ai/Yi-6B/blob/main/modeling_yi.py & https://huggingface.co/01-ai/Yi-6B/blob/main/config.json, and the technique report is in preparation.

from yi.

If you don't mind a follow-up question, this is just a base model, so there's no prompt format or anything like that to take into consideration. Is that correct?

I was able to successfully quantize it to GGUF and run inference but the results I got are a bit strange.

Yes, it's just a base model. What's your prompt? We can study this case together.

from yi.

Thanks for the answer! If it's just the names that are different compared to LLaMA2 then it should be pretty easy to add support in llama.cpp (my main interest). GGUF support should help your model get more exposure as well.

from yi.

If you don't mind a follow-up question, this is just a base model, so there's no prompt format or anything like that to take into consideration. Is that correct?

I was able to successfully quantize it to GGUF and run inference but the results I got are a bit strange.

from yi.

What's your prompt? We can study this case together.

Thanks for the reply. I'll make a question in Discussions instead of continuing here so you can keep your issues on topic. (This one can be closed if you want.)

(Edit)
For further discussion: #5

from yi.