Questions about the pSp encoder with Z+ about dualstylegan HOT 4 CLOSED

1. Could you share how you modified pSp encoder with Z+?
   I could not find the details in the paper and codes.
   It seems you didn't use VAE encoder of the AgileGAN and also didn't inference Z mean and Z standard.
   The modification of pSp encoder is only passing output (Z+) of pSp encdoer to mapping layer indviually? Am I right?

Yes, the encoder just maps the input image to Z+ output without using VAE.

2. How did you train the pSp encoder with Z+? Or you used the pretrained pSp encoder model directly?

(I) I modify the stylegan model (pSp decoder) so that it accepts Z+ input:

(2) I modify pSp encoder so that it passes the options z_plus_latent and return_z_plus_latent to the pSp decoder

(3) I modify the function train() and validate() of coach.py in the official pSp code, so that it passes our training option opts.z_plus_latent to the pSp model.

• changing Line 91 to y_hat, latent = self.net.forward(x, return_latents=True, z_plus_latent=self.opts.z_plus_latent)
• changing Line 135 to y_hat, latent = self.net.forward(x, return_latents=True, z_plus_latent=self.opts.z_plus_latent)

(4) Finally, I train pSp from scratch using the following training settings:

python scripts/train.py \
--dataset_type=ffhq_encode \
--exp_dir=./logs/ffhq_encoder_zplus/ \
--workers=4 \
--batch_size=8 \
--test_batch_size=8 \
--test_workers=8 \
--val_interval=2500 \
--save_interval=5000 \
--encoder_type=GradualStyleEncoder \
--start_from_latent_avg \
--lpips_lambda=0.8 \
--l2_lambda=1 \
--id_lambda=0.1 \
--z_plus_latent

Hope this solves your questions.

from dualstylegan.

Thank you for your clear reply!

from dualstylegan.

@williamyang1991 are the changes you suggested to the psp decoder (stylegan) and psp code both to be made in the original psp code repo and then train ? In order to obtain the z plus encoder

from dualstylegan.

@williamyang1991 are the changes you suggested to the psp decoder (stylegan) and psp code both to be made in the original psp code repo and then train ? In order to obtain the z plus encoder

Yes, I modify both the encoder and decoder.

from dualstylegan.