Please, add local LVM model support and example [feature request] about comfyui_llm_party HOT 8 CLOSED

When I have time, I will start working on LVM-related tasks, beginning with the LLaVA model :)

from comfyui_llm_party.

That’s really great, I also think that supporting LVM is an indispensable feature.

from comfyui_llm_party.

I’ve been a bit busy this past week, mainly spending time creating a tutorial video for this project and fixing some minor issues within the project. It was only during the weekend that I had a substantial amount of time. May I ask if you have already started writing the PR related to the LVM model? If so, I won’t duplicate the development. Instead, I plan to create something that can package the comfyui workflow into an OpenAI interface, making it convenient for any software that can integrate with OpenAI to access user-defined workflows. If you haven’t started yet, I can begin writing one, and you can provide your feedback on it later.

from comfyui_llm_party.

Unfortunately, I had exactly the same situation, and I didn’t have time all week. And I expecting next week will be full of work too. :(
On this topic, I was only able to check the nodes from here https://github.com/gokayfem/ComfyUI_VLM_nodes and the nodes for the local version of Ollama - https://github.com/stavsap/comfyui-ollama

I can say that specifically for those situations where we use ComfyUI, the approach with another local server is not suitable, since then control over unloading the model is lost and Comfi’s workflows end up waiting for the computer with the ollama server to process the request.
And this adds the requirement to either have a second computer or Ollama to have to run on the CPU if a person has only one computer.

The way you described with a universal interface, when you can outsource some of the tasks to another service, is quite an interesting and good solution, imho :)

from comfyui_llm_party.

Sorry for the wait, I have adapted this model: llava-llama-3-8b-v1_1-gguf. The example workflow can be found here: start_with_LVM.json.Due to the use of llama_cpp_python code, it may not be perfectly compatible with MPS devices. You can see the adaptation code I made for different devices here. I’m not sure if it will cause errors on macOS and MPS, and I would greatly appreciate your help. 🙂

from comfyui_llm_party.

llama_cpp_python is working fine, there are just a problem with installing it from default Python's pipy(but it is available in GH releases).

Surprisingly, it worked with int4 - I didn’t even have to do anything :)

Good job, really.

Issue now can be closed?

from comfyui_llm_party.

Just a note: will be good not to display a encoded image data in "history"

A few big images with 10-25 MB will make history totally non displayable in browser.

from comfyui_llm_party.

The issue with the history has been resolved, and the download source for llama_cpp_python in macOS has been adjusted. The problem is now solved.I sincerely appreciate your assistance. Should you have any further recommendations or if there’s anything else I might require, I would be grateful if you could kindly reach out to me.：)

from comfyui_llm_party.