Code Monkey home page Code Monkey logo

anynode's Introduction

AnyNode v0.1 (๐Ÿ„ beta)

A ComfyUI Node that uses the power of LLMs to do anything with your input to make any type of output.

image

image

Watch the video

Install

  1. Clone this repository into comfy/custom_nodes (sorry, not on Manager just yet!)
  2. Make sure you have the openai module installed through pip: pip install openai
  3. Add your OPENAI_API_KEY variable to your Environment Variables. How to get your OpenAI API key
  4. Restart Comfy
  5. In ComfyUI double-click and search for AnyNode or you can find it in Nodes > utils

How it Works

  1. Put in what you want the node to do with the input and output.
  2. Connect it up to anything on both sides
  3. Hit Queue Prompt in ComfyUI

AnyNode codes a python function based on your request and whatever input you connect to it to generate the output you requested which you can then connect to compatible nodes.

image

Warning: Because of the ability to link ANY node, you have to make sure it nails the output.

Caveats

  • I have no idea how far you can take this nor it's limits
  • LLMs can't read your mind. To make complex stuff in one node you'd have to know a bit about programming
  • Right now you can only see code the LLM generates in the console
  • Currently there is no way for it to "edit" the code but that is coming soon
  • You currently need an OpenAI API Key
  • Can't make a sandwich

Strengths

  • You can use as many of these as you want in your workflow creating possibly complex node groups
  • Really great at single purpose nodes
  • Uses OpenAI API for simple access to the latest and greatest in generation models
  • Technically you could point this at vLLM or Ollama for you LocalLLM fans
  • Can use most of the popular python libraries and most standard like (numpy, torch, collections, re)
  • Ability to make more complex nodes that use inputs like MODEL, VAE and CLIP with input type awareness

Coming Soon

  • Saving the generated functions in your workflows
  • Multiple Inputs and outputs
  • Error Mitigation: Auto-correct errors
  • Incremental Editing (Using the previously generated code as reference to your instruction)

If you're still here

Let's enjoy some stuff I made while up all night!

image This one, well... the prompts explain it all, but TLDR; It takes an image as input and outputs only the red channel of that image.

Screenshot from 2024-05-26 01-30-40 Here I use three AnyNodes: One to load a file, one to summarize the text in that file, and the other to just do some parsing of that text. No coding needed.

image I took that Ant example a bit further and added in the normal nodes to do img2img with my color transforms from AnyNode

anynode's People

Contributors

newsbubbles avatar lks-ai avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.