Comments (6)
Please let me know if this feature is ever implemented. Thanks
We provide an offline-compile mode to reduce online-compile time for scenarios where the input shapes of online inference is limited. Hope this will help: https://github.com/Oneflow-Inc/diffusers/wiki/How-to-Run-OneFlow-Stable-Diffusion#optimization-for-multi-resolution-picture
from onediff.
Compiling a dynamic shape graph is not supported right now. We use some optimization skills which are related to static input shape:
- memory static allocation;
- constant folding;
Have you met some problems with static shapes?
from onediff.
Have you met some problems with static shapes?
No, but we are building a service that plans to offer dynamic resolutions.
I was thinking of compiling it for multiple resolutions, but that would take up vram space
from onediff.
Please let me know if this feature is ever implemented. Thanks
from onediff.
Please let me know if this feature is ever implemented. Thanks
We provide an offline-compile mode to reduce online-compile time for scenarios where the input shapes of online inference is limited. Hope this will help: https://github.com/Oneflow-Inc/diffusers/wiki/How-to-Run-OneFlow-Stable-Diffusion#optimization-for-multi-resolution-picture
Thanks. First I call pipe.enable_graph_share_mem()
, then run inference on the resolutions I want, and the graph should be ready for those resolutions right?
pipe.enable_graph_share_mem()
prompt = "a photo of an astronaut riding a horse on mars, red sky, (green sky:1.5)"
with torch.autocast("cuda"):
images = pipe(prompt, height=1024).images
images = pipe(prompt, height=768).images
images = pipe(prompt, height=512).images
images = pipe(prompt, height=256).images
sorting the input shape from large to small to trigger graph compilation
I see that it takes 1 second to load when it changes resolution:
images = pipe(prompt, height=256).images
images = pipe(prompt, height=256).images
images = pipe(prompt, height=512).images
images = pipe(prompt, height=512).images
images = pipe(prompt, height=768).images
images = pipe(prompt, height=768).images
images = pipe(prompt, height=1024).images
images = pipe(prompt, height=1024).images
images = pipe(prompt, height=768).images
images = pipe(prompt, height=768).images
images = pipe(prompt, height=512).images
images = pipe(prompt, height=512).images
images = pipe(prompt, height=256).images
images = pipe(prompt, height=256).images
Is there anyway to speed this up? or maybe mantain all of them active? I don't mind having to use more vram.
from onediff.
You can run and read this test to be familiar with these two features: https://github.com/Oneflow-Inc/diffusers/blob/oneflow-fork/tests/test_pipelines_oneflow_graph_load.py
from onediff.
Related Issues (20)
- StableVideoDiffusionPipeline 加速 V100 能执行成功,A30 执行报错 HOT 14
- 开启onediff优化后,A100上unet执行结果为nan,但是V100上unet计算结果正常 HOT 5
- Almost no speed up on L4 HOT 3
- how to use bf16 for onediff HOT 2
- > 复现步骤: 1.首次工作流中使用OneDiffCheckpointLoaderSimple加载ckpt,且不加载任何Lora 2.修改工作流,增加LoRALoader节点,加载任一Lora 3.LoRA未生效 HOT 4
- Enterprise quant has problem
- It's weird if both instant_id and ip_adapter are enabled in comfyui
- Compiled VAE encoder returns NaN
- OP Kernel Not Found Error HOT 1
- Cannot run sdxl sample code with deepcache, PyTorch should be >= 1.12 HOT 3
- no upcast_vae() for SVD pipe HOT 1
- How to use int8 for onediff Controlnet HOT 5
- 加载lora->出图->卸载lora 重复多次后,质量降低 HOT 7
- How to TRAIN the int8 quant model? HOT 5
- Issue with using oneflow as well as with nexfort HOT 1
- onediff通过劫持torch导致的额外编译时间 HOT 1
- 订阅企业版后是否会提供源码? HOT 1
- Kaggle Notebook installation HOT 10
- when using load_pipe(pipe, dir="cached_pipe"), if the image size does not match the cached image size, an error occurs. Reshape infered output element count is different with input in op_name: model.mid_block.attentions.0-reshape-9 input shape is : (1,512,80,80) , output shape is : (1,512,5120) , output logical shape is (1,512,5120) , and reshape shape conf is : (1,512,5120). HOT 3
- oneflow_compile(unet) failed HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from onediff.