llama3 template about gpt4all HOT 5 OPEN

I tested out newer Llama 3s made with the latest llama.cpp and they do have issues like showing formatting and talking past the end token when using GPT4All. And supporting them would be a very nice bonus because they're notably more coherent and less buggy after recent fixes. For example, they can solve 3333+777, rather than respond with 33 + 77 = 101.

This is the answer GPT4All v2.7.4 with the including L38 Instruct Q4_0 gives.

"Let me calculate the sum for you...

33 + 33 = 66
66 + 77 = 143

So, the answer is: 143. Is there anything else I can help you with?"

from gpt4all.

these issues have been fixed in llama.cpp but the lama.cpp fork of gpt4all has not been updated so far. There are also some speed improvements for prompt processing which hopefully will also be made available in gpt4all.

from gpt4all.

@Phil209 about formatting issues, have you encountered the following problem:

ERROR: byte not found in vocab: '
'

from gpt4all.

@agilebean No, I've never seen anything like "ERROR: byte not found in vocab:" before.

The formatting being shown is the standard stuff after the end token, such as "###System...", followed by various things like a potential user response, followed by what the assistant should then say..., or related examples, or an interesting related fact, or instructions for how it should responsibly respond as an AI, and so on.

from gpt4all.

This is off topic, but I'd just like to say thanks for the Q4_0 of Llama 3 8b Instruct you provided. I used various apps to test other Q4_K_M or Q5_K_M versions assuming that they would be better, but your smaller Q4_0 in GPT4ALL performed the best. For example, the responses when asking for a list of main characters, and the actors who portrayed, reliably had fewer hallucinations.

from gpt4all.