Structured Output breaking. about comfyui_vlm_nodes HOT 6 OPEN

!!! Exception during processing !!!
Traceback (most recent call last):
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\AI\ComfyUI\ComfyUI\custom_nodes\ComfyUI_VLM_nodes\nodes\suggest.py", line 611, in keyword_extract
parsed_response = json.loads(response)
^^^^^^^^^^^^^^^^^^^^
File "json_init_.py", line 346, in loads
File "json\decoder.py", line 337, in decode
File "json\decoder.py", line 353, in raw_decode
json.decoder.JSONDecodeError: Unterminated string starting at: line 3 column 15 (char 17)

from comfyui_vlm_nodes.

Over and over and over. keeps breaking workflows and preventing continuous processing without babysitting the node.

from comfyui_vlm_nodes.

which llm you are using?

from comfyui_vlm_nodes.

llava 1.6 7b.Q5 mistral

from comfyui_vlm_nodes.

maybe you can use more json output oriented llm, like nous hermes type llms, you can find their gguf files in huggingface

from comfyui_vlm_nodes.

Unfortunately the hermes models kind of suck for this purpose, the llava base models give significantly better outputs, any chance of adding some basic formatting error handling into the node?

from comfyui_vlm_nodes.