experiments with throwing many images at decoder (via ffmpeg) about pti HOT 4 CLOSED

Do you mean tuning the generator based on different images of the same person from a video?

from pti.

yes. - one person. or is that not going to work?

(I think I've worked out the pkl problem - I'm using the e4e generator to inject in - I need to swap in the pti generator and save that instead. I just haven't gotten around to it. I also have doubts with the g_ema findings....)

from pti.

The thing about StyleGAN's latent space which I have noticed while researching the inversion problem is that if you have the same identity, but with different backgrounds - the corresponding latent codes will reside in different places in the W space.
Hence, I think that using the same person and the same background will result in even better reconstruction (maybe it will enable the tuning to be faster?)
For the same person and different backgrounds, PTI will handle it like 2 different identities.

Most of the time in a video the background remains the same, so it seems PTI will be even better.

Notice one thing though - if you take the first frame and receive a new tuned StyleGAN using PTI. it will already contain the identity you want. Meaning the 10th frame will already be in the W space even if you never used PTI on it. You can imagine the difference between the 10th frame and the first frame to be an edit. This explains why every image of the same identity and background is already in the tuned generator and does not require new tuning.

Hope it answers your question, I have wondered a bit in my answer to explain even more stuff which might be helpful

from pti.

fyi - http://arxiv.org/abs/2201.08361

from pti.