Raw approach runs into odd response about node-llama-cpp HOT 1 CLOSED

In the raw approach, you have to set the condition to stop generating output by yourself; unless you stop it from generating more text, it can go on forever.

According to the code you shared, since your model never outputted the USER: string, it didn't stop.
You can try to ask the model to write some end-sequence text when it finishes generating the wanted output, and when you encounter this end-sequence text, stop the generation.
You can also stop the generation when the output text reaches a certain length.

I suggest you use LlamaChatSession as it overcomes those challenges, and if you need to use some special wrapping around your prompts, you can implement your own ChatPromptWrapper.

from node-llama-cpp.