Stable Diffusion Deep Dive fails at UNET and CFG with IndexError: index 51 is out of bounds for dimension 0 with size 51 about diffusion-nbs HOT 4 CLOSED

Okay for those who are curious, I was able to fix it by making this change in the code, I think we were doing step twice in the same loop and it should be done once per loop right? correct me if I'm wrong here @johnowhitaker

Earlier code:

# Get the predicted x0:
latents_x0 = scheduler.step(noise_pred, t, latents).pred_original_sample # Using the scheduler (Diffusers 0.4 and above)

# compute the previous noisy sample x_t -> x_t-1
latents = scheduler.step(noise_pred, t, latents).prev_sample

Modified code (call step only once and use intermediate variable scheduler_output):

scheduler_output = scheduler.step(noise_pred, t, latents)

latents_x0 = scheduler_output.pred_original_sample # Using the scheduler (Diffusers 0.4 and above)

# compute the previous noisy sample x_t -> x_t-1
latents = scheduler_output.prev_sample

from diffusion-nbs.

Hmm, I can't re-create. If it's failing at the last step you could stop a few steps early (if i == 45: break) and you'll get to see the animation even if it isn't ideal.
Could you confirm that a fresh copy of the notebook fails here when you do 'Run all'?

from diffusion-nbs.

workaround

        latents = latents.detach() - cond_grad * sigma**2
        scheduler._step_index = scheduler._step_index - 1

or uncomment the manual stepping instead scheduler stepping

# Get the predicted x0:
latents_x0 = latents - sigma * noise_pred
# latents_x0 = scheduler.step(noise_pred, t, latents).pred_original_sample

from diffusion-nbs.

yeah getting the same error with index at UNET and CFG sections in the code to generate the video.. I tried reducing num_inference_steps to 45 but same error happens saying index 46 is out of bounds..

@venkyyuvy could you explain how exactly to use the first part of your workaround? I don't think cond_grad is defined in the code [nvm, found it in CFG section].. regarding the uncommenting part, I feel doing it through API (step) would be better than manually doing it like that right?

from diffusion-nbs.