Add problem solving to evol-instruct about wizardlm HOT 3 OPEN

Thanks for your valuable suggestion, we found that the skills you mentioned improved when finetune with the larger llama model (i.e., 13B). We will continue to think about new ideas to improve these skills.

from wizardlm.

Thank you for the timely response. I'd be interested to see how well the 13B model performed on these questions, which I can't do since I only have 8GB of RAM and a pretty weak CPU, only being able to play with the model on Gradio or through LLaMA.cpp.

Still, I find it fascinating to see how projects like this push the limits of what's possible with that low of a parameter count, prompting even the attention of Google and Microsoft (referring to Google's "we have no moat" memo and Microsoft's TinyStories experiment). I wonder if any meaningful results on this complex task can be achieved at just 7B without even training a model from scratch.

from wizardlm.

The newly released WizardLM 13B, which dataset included more physics questions, had finally started forming coherent reasoning chains, correctly doing basic calculations, rearranging equations, and solves simple problems as well as gpt-3.5, which Guanaco 65B couldn't achieve.

However, interestingly, WizardLM 30B consistently hallucinates an incorrect reasoning chain, giving us snowballing hallucinations that end up with an incorrect answer. Perhaps this can give us some insight into effective scaling and training settings for a given dataset and foundation model.

from wizardlm.