Add VAE to txt-to-speech Inference about docker-diffusers-api HOT 5 OPEN

Yup! negative_prompt modelInput, as it seems you worked out.

The modelInput's are passed directly to the relevant diffusers' pipeline, so you can use whatever arguments are supported by that pipeline. I made this a little clearer in the README a few days ago with links to the common diffusers pipelines, as I admit it wasn't so obvious until then 😅

There's also a note there now about using the lpw_stable_diffusion pipeline which supports longer prompts and prompt weights.

Thanks for all the kind words! 🙌

from docker-diffusers-api.

Hey, @digiphd! Thanks for getting this on my radar. I'll have a chance to take a look during this coming week.

As a preliminary comment, I like the idea of being able to switch the VAE at runtime, although there will be a lot of work involved to adapt how we currently cache models.

P.S. If you're impatient, in the meantime, I think you could probably:

1. Clone https://huggingface.co/runwayml/stable-diffusion-v1-5/tree/fp16
2. Replace the vae directory with the contents from https://huggingface.co/stabilityai/sd-vae-ft-mse/tree/main
3. Upload that "new" model back to HuggingFace and build docker-diffusers-api with that (it's possible without uploading back to huggingface, but a bit more complicated).

Alternatively, with your current setup, it's possible that if you set MODEL_PRECISION="" and MODEL_REVISION="", you might get past that error by using full precision (but inference will be slower; nevertheless, maybe something useful in the interim).

Anyways, have a great weekend and we'll be in touch next week 😀

from docker-diffusers-api.

Hey @gadicc great, thanks for your suggestions I will give them ago! You're a legend!

Another thing I was wondering, was if docker-diffusers-api text-to-image supports negative keywords?

I did put it as an argument and it seemed to negatively affect the output images.

from docker-diffusers-api.

Hey @digiphd, I had a quick moment to try dreamlike-art/dreamlike-photoreal-2.0 and it works out the box for me, in both full and half precision. What version of docker-diffusers-api are you using?

These worked for me:

$ python test.py txt2img --call-arg MODEL_ID="dreamlike-art/dreamlike-photoreal-2.0" --call-arg MODEL_PRECISION=""
$ python test.py txt2img --call-arg MODEL_ID="dreamlike-art/dreamlike-photoreal-2.0" --call-arg MODEL_PRECISION="fp16"

I just tried in the default "runtime" config. If you have this issue specifically in the -build-download variant, let me know.

from docker-diffusers-api.

Related: #26

from docker-diffusers-api.