Comments (7)
there is now 2 new nodes for llama.cpp
LLava Optional Memory Free Simple
LLava Optional Memory Free Advanced
You can use multiple of this nodes in your workflow.
from comfyui_vlm_nodes.
working on that right now. i managed to do it for transformers models. i will make it possible for llama.cpp models also. there will be an option to keep in memory or unload it soon.
from comfyui_vlm_nodes.
You mean, you want to use it as an llm. I can add LLM memory free version of this node also.
from comfyui_vlm_nodes.
there is already llm loader and llm sampler and you can use llava models as llm on them. they are currently not supporting unloading option but i will add llm versions of them also.
from comfyui_vlm_nodes.
Perfect! Thank you!
Works great!
The only thing is that I would like to have a node option (or an image input parameter option) without the mandatory submission of an image as an input, if the user only plans to generate text on prompt.
Since in the current version of the node, I am forced, firstly, to specify an image, which can spoil the generation result, and secondly, I am forced to specify a much larger max_ctx size (at the moment I now need to specify a value twice as large for my tasks (4096) as before without necessarily adding an image), increasing the load/generation time.
Is it possible to make the "image" input parameter optional?
Or tell me how I can modify the code for the LLava Optional Memory Free Advanced node to achieve this behavior?
Thanks for your hard work and help!
from comfyui_vlm_nodes.
Yes, like an LLM, that's right. It will be very cool to have such a node.
Thanks again!
from comfyui_vlm_nodes.
BUG:
Memory leak issue when using node LLava Optional Memory Free.
LLava Optional Memory Free does not unload the CLIP model from memory. At the same time, with each launch of the queue, the CLIP model is loaded into additional memory in VRAM, rather than overwriting the previously loaded one (does not use the previously loaded one) into memory. That is, after several launches I get video memory overflow and, accordingly, a drop in performance.
You can track this by the VRAM volumes during generation.
I hope there is a solution for this!
LLM Optional Memory Free would be extremely useful for many tasks, I’ll be waiting, thanks for your work! 💪
from comfyui_vlm_nodes.
Related Issues (20)
- GPU not utilized with standard llama-cpp-python HOT 1
- Error using xcomposer: Can't move a model that's been offloaded (delete this please)
- Get Keywords bug HOT 2
- Moondream error after installing VLM nodes HOT 2
- IMPORT FAILED in ComfyUI HOT 2
- Structured Output breaking. HOT 6
- LLaVA++ HOT 1
- ValueError("Prompt exceeds n_ctx") # TODO: Fix HOT 6
- RuntimeError: Failed to load shared library 'D:\AI\ComfyUI_windows_portable\python_embeded\Lib\site-packages\llama_cpp\llama.dll HOT 2
- Using Llama3 and other models via Replicate HOT 2
- Load the wrong GGUF model causes (core dump) crash
- Kosmos 2.5 released - Possible to add support for it? HOT 2
- requesting deepseek-vl and qwen-vl nodes HOT 5
- When huggingface-hub==0.20.3 is forcibly installed, ComfyUI fails to start. HOT 4
- Is it possible to combine both branches (main + mac) into the one? HOT 2
- Can't load examples or open/save workflow images
- Moondream issue HOT 6
- ValueError: could not broadcast input array from shape (32000,) into shape (0,) HOT 1
- Add ability to perform OpenAI API compatible calls? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from comfyui_vlm_nodes.