Dynamic Resolution Compilation about onediff HOT 6 CLOSED

Please let me know if this feature is ever implemented. Thanks

We provide an offline-compile mode to reduce online-compile time for scenarios where the input shapes of online inference is limited. Hope this will help: https://github.com/Oneflow-Inc/diffusers/wiki/How-to-Run-OneFlow-Stable-Diffusion#optimization-for-multi-resolution-picture

@chavinlo

from onediff.

Compiling a dynamic shape graph is not supported right now. We use some optimization skills which are related to static input shape:

• memory static allocation;
• constant folding;

Have you met some problems with static shapes?

from onediff.

Have you met some problems with static shapes?

No, but we are building a service that plans to offer dynamic resolutions.
I was thinking of compiling it for multiple resolutions, but that would take up vram space

from onediff.

Please let me know if this feature is ever implemented. Thanks

from onediff.

Please let me know if this feature is ever implemented. Thanks

We provide an offline-compile mode to reduce online-compile time for scenarios where the input shapes of online inference is limited. Hope this will help: https://github.com/Oneflow-Inc/diffusers/wiki/How-to-Run-OneFlow-Stable-Diffusion#optimization-for-multi-resolution-picture

@chavinlo

Thanks. First I call pipe.enable_graph_share_mem(), then run inference on the resolutions I want, and the graph should be ready for those resolutions right?

pipe.enable_graph_share_mem()

prompt = "a photo of an astronaut riding a horse on mars, red sky, (green sky:1.5)"
with torch.autocast("cuda"):
    images = pipe(prompt, height=1024).images
    images = pipe(prompt, height=768).images
    images = pipe(prompt, height=512).images
    images = pipe(prompt, height=256).images

sorting the input shape from large to small to trigger graph compilation

I see that it takes 1 second to load when it changes resolution:

    images = pipe(prompt, height=256).images
    images = pipe(prompt, height=256).images
    images = pipe(prompt, height=512).images
    images = pipe(prompt, height=512).images
    images = pipe(prompt, height=768).images
    images = pipe(prompt, height=768).images
    images = pipe(prompt, height=1024).images
    images = pipe(prompt, height=1024).images
    images = pipe(prompt, height=768).images
    images = pipe(prompt, height=768).images
    images = pipe(prompt, height=512).images
    images = pipe(prompt, height=512).images
    images = pipe(prompt, height=256).images
    images = pipe(prompt, height=256).images

Is there anyway to speed this up? or maybe mantain all of them active? I don't mind having to use more vram.

from onediff.

You can run and read this test to be familiar with these two features: https://github.com/Oneflow-Inc/diffusers/blob/oneflow-fork/tests/test_pipelines_oneflow_graph_load.py

• pipe.enable_graph_share_mem()
• pipe.save_graph() and pipe.load_graph()

@chavinlo

from onediff.