gnobitab / instaflow Goto Github PK

Hi! Thanks for the awesome paper and impressive results! I have one question about the training process.
When we are predicting velocity in say sd1.4, the usual objective is just epsilon but do you do some transformation to make it v or do you have the network predict v immediately? Thanks!
Also, I made a blog here talking about the paper+plans to make it a diffuser pr. The plan I currently have is

• Make a script to generate the latent noise, images, and text to save to the dataset
• Make rectified flow/reflow script
• Make distillation script

hello

for example the Würstchen v2 is 42x spatial compression

This is a very interesting project and I look forward to the release of the code

We have converted both models to ONNX format and have added initial support for InstaFlow to OnnxStack

ONNX Model
https://huggingface.co/TheyCallMeHex/InstaFlow-0.9B-ONNX/tree/main
https://huggingface.co/TheyCallMeHex/2-Rectified-Flow-ONNX

OnnxStack UI download (windows only)
https://github.com/saddam213/OnnxStack/releases/tag/v0.9.1-pre

InstaFlow-0.9B | 1 Step

When I access https://huggingface.co/spaces/XCLiu/InstaFlow, it shows Runtime error

Hello,

I am specifically interested in applying this method to optimize a diffusion model trained on clip vision embeddings, specifically Stable Diffusion Reimagine which is a version of SD 2.1. Are there any plans to train an InstaFlow model adapting this model or another model that uses vision embeddings rather than text embeddings?

Are the models compatible with DDIM inversion? or just plain latents = latents - dt * v_pred for the forward diffusion? Ive tried and didnt get good results. Making sure if it is a bug from my end.

I am god.
As title says!
I am very dissatisfied.
Yesterday you release it, but YOU LIED TO OPEN SOURCE COMMUNITY!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
We need this TODAY!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
I don't have any more ! keys left to throw at you.
I AM CRYING FOR YOU :(
this is godly order:::: RELEASE MODEL IN MIT LICENSE TODAY!

Thanks and congratulations on your excellent work!

Will the InstaFlow-0.9B pre trained weights be released?

Can this method used for inpainting？

1. Docker image does not work. Gives me this error: https://pastebin.com/kE6TLYju . Seems like the repo is just broken. Could you please fix it?
2. Is it a tool for making a proper inference of SD 1.5 in 1 step or it's just a model different from SD 1.5 which is able to generate images in 1 step? Did not get this from your GitHub

I specialize in SD performance. I have a 4090 on a i9-13900K system running Ubuntu 22.04. Some of the optimizations I've discovered have been incorporated into pytorch and diffusers.

I saw your recent Sep 12 InstaFlow announcentment. A github without code???
It refers to another github repro with was created 10 months ago! How can this do txt2img 3X faster than regular SD and not been noticed for 10 months?

Sorry for posting this as an "issue" but this github repro has no "discussion" section.
I evaluate and consult on SD performance. I have gotten over 90+ it/s with TensorRT and have tested torch.compile() and AOT. I can average under .4 seconds per image with standard A1111 generations with batching and advanced tuning.

While I am concerned that your code is so old, so I'm not sure if I am wasting my time, I have just cloned your RectifiedFlow. Hmmm, I see the README has what is supposed to be a "High-Resolution Generation" EXAMPLE yet nowhere does the word "prompt" occur and I have also checked main.py. So I'm not sure how to even try this.

THIS IS AN ORDER, EFFECTIVE NOW,
AS TITLE SAYS
DO NOT WAIT ANYMORE!!!!
I (I AM GOD BTW!!!)) so where i was... I AM DIRECTLY GIVING THIS AS A ORDER!!!

When I run https://github.com/gnobitab/InstaFlow/tree/main/code#inference-instaflow-09b-one-step-generation, I meet the following error.

Traceback (most recent call last):
File "/dfs/comicai/songtao.tian/InstaFlow-main/code/test_instaflow.py", line 4, in
pipe = RectifiedFlowPipeline.from_pretrained("/dfs/comicai/zhiyuan.shi/models/XCLiu/instaflow_0_9B_from_sd_1_5", torch_dtype=torch.float16)
File "/root/miniconda3/envs/LCM/lib/python3.9/site-packages/huggingface_hub/utils/_validators.py", line 118, in _inner_fn
return fn(*args, **kwargs)
File "/root/miniconda3/envs/LCM/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 1271, in from_pretrained
loaded_sub_model = load_sub_model(
File "/root/miniconda3/envs/LCM/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 525, in load_sub_model
loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
File "/root/miniconda3/envs/LCM/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2028, in from_pretrained
return cls._from_pretrained(
File "/root/miniconda3/envs/LCM/lib/python3.9/site-packages/transformers/tokenization_utils_base.py", line 2260, in _from_pretrained
tokenizer = cls(*init_inputs, **init_kwargs)
File "/root/miniconda3/envs/LCM/lib/python3.9/site-packages/transformers/models/clip/tokenization_clip.py", line 327, in init
self.encoder = json.load(vocab_handle)
File "/root/miniconda3/envs/LCM/lib/python3.9/json/init.py", line 293, in load
return loads(fp.read(),
File "/root/miniconda3/envs/LCM/lib/python3.9/json/init.py", line 346, in loads
return _default_decoder.decode(s)
File "/root/miniconda3/envs/LCM/lib/python3.9/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/root/miniconda3/envs/LCM/lib/python3.9/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Unterminated string starting at: line 48921 column 3 (char 1041097)

I'm amazed by the performance of rectified flow and its series And there is no training code of it in instaflow. So why? I mean there's one in rectified flow and since instaflow is a later work Produced by the same group It must have some change in the training code like in the reflow. I mean I can write a training code according to the paper but I'm afraid it may not be quite the same as the official one. And I want to do some comparison. So could someone help to other one?
I would be appreciate if someone can do Thanks you for your time and happy new year

The reasons are as follows:
I am a worker in the field of the acceleration inference step in text-to-image diffuison, the similarity works such as Snapfusion and W-condition model are not open source currently.
Snapfusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds,https://github.com/snap-research/SnapFusion
W-condition: On Distillation of Guided Diffusion Models (CVPR 2023, Award candidate).

If the author does not intend to open source the code, I will reproduce the InstaFlow. Could you please communicate with me about the InstaFlow.

I, GOD, HAVE TO SAY!!!! I AM VERY HAPPY FOR CODE IN MIT LICENSE <3
@gnobitab , @lqiang67 good job !!!

During reflow or distill， does the {1、2}_rectified_model or distill model need to intergrate the text guidance information。Such as the guidance scale alpha=6 during reflow, and alpha=1.5 during distill

I cant train a model, i only got a 1070ti, I BEG YOU TO RELEASE COMMERCIAL MODEL TODAY!!!!!!! OR I WILL OFF MYSELF I AM

Imperative steps necessary to release the code:
FIRST ORDER: Release the Code
SECOND ORDER: Release ALL models
THIRD ORDER: LiCENSE MUST BE MIT OR SIMILAR COMMERCIAL FREE USE!!!!
Thanks from the open source community <3

@gnobitab THIS IS THE FORM OF CODE RELASING! There is no other viable or "open source" form as mentioned in #2