The inputs of view-conditioned generation. about magicdrive HOT 4 CLOSED

Yes, we use DDIM inversion.

from magicdrive.

I see. Then, what is the diffusion model used to conduct the inversion? From my understanding, if the single-image diffusion model is used, it cannot faithfully reproduce the conditional view, as the final DDIM sampling is processed by the multi-view diffusion model. Am I right?

from magicdrive.

Sorry, I made a mistake in the last reply. We did not use DDIM inversion. We

1. add the sampled noise to the given view according to the scheduler and use it as the input for noise prediction;
2. replace the predicted noise with ground truth for the given view.

similar to the inpainting process.

from magicdrive.

Sounds effective. Thanks for your detailed reply.

from magicdrive.