loadResources took long time to load UNET about ml-stable-diffusion HOT 6 OPEN

Are you using the CPU+GPU or the Neural Engine for inference? When using the Neural Engine, it takes a really long time to re-compile the model. On your MacBook Pro, you should be able to use the CPU+GPU option which does not need recompilation.

from ml-stable-diffusion.

@MenKuch
After I set computeUnits = .cpuAndGPU, it only take 16s to start the app, thanks!

https://www.photoroom.com/inside-photoroom/core-ml-performance-2022#

ANE + CPU is much more faster than GPU + CPU (about 500% increase), so how can we use it?
we are using coreml-stable-diffusion-v1-5 from huggingface, it is coreml model already, why does it need to re-compile the modal when running on ANE config, can we manual convert it? Many thanks!

from ml-stable-diffusion.

In my understanding, the initial compilation is necessary because the model has to be compiled for a specific ANE generation (i.e. the Neural Engine for the A14, A15, A16, M1 and so on)

from ml-stable-diffusion.

Is there anyway to speed up the loading when using .cpuAndNeuralEngine?
or other format not mlmodelc? It's not acceptable to compiled a model in 6 mins.....

from ml-stable-diffusion.

To my knowledge, there is currently no way of speeding up the process for a model. Worse: You seem to cannot get the progress of the conversion process so there is no way of letting the user know how long he has to wait.

from ml-stable-diffusion.

Ok, thank you!

from ml-stable-diffusion.