Pretrained model about nonlinear_face_3dmm HOT 12 OPEN

seems input is 224x224, and the ouput is not very good

from nonlinear_face_3dmm.

@yuellong Thank you can you help me turn it on? I am trying to run it on Google Collab? Please can you please guide me with the required steps

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

from nonlinear_face_3dmm.

Can you tell me how you were able to run the pre-trained model, and what steps you took, please @cassiePython

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face


            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5
            
            
            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)
            
            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here:

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face


            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5
            
            
            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)
            
            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here:

Nonlinear_Face_3DMM/model_non_linear_3DMM_proxy.py

Line 445 in 662098a

if self.is_using_frecon:

@HOMGH Thank you very much!
I have another question. Do I need to modify the variables G_images_mix_s1a0_300W_vgg and G_images_mix_s0a1_300W_vgg in this code?
I once tried to imitate the model_non_linear_proxy.py file to write the G_loss_frecon code, but when I ran it, the system reported an error.

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face


            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5
            
            
            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)
            
            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here:

Nonlinear_Face_3DMM/model_non_linear_3DMM_proxy.py

Line 445 in 662098a

if self.is_using_frecon:

@HOMGH Thank you very much!
I have another question. Do I need to modify the variables G_images_mix_s1a0_300W_vgg and G_images_mix_s0a1_300W_vgg in this code?

No, I don't think so. Actually I didn't get good results after training for 40 epochs which took me around 20 days! So I'm not sure whether this code works or not!

from nonlinear_face_3dmm.

I run the pre-trained model but get unreasonable outputs. The shape can not be visualized with 3DMesh

Same issue! The reconstructed texture (rendered image) is not correct as well!

Hi，@HOMGH Can you tell me how to write the G_loss_frecon code? Thank you

Hi, this is the part of the code which compute G_loss_frecon:

        if self.is_using_frecon:
            print('is calculating frecon.....')
            from vgg_face import vgg_face


            #G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask)+ 1.0)*127.5
            G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0)*127.5
            G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0)*127.5
            input_images_300W_vgg      = (input_images_300W      + 1.0)*127.5
            
            
            input_features, _, _  = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_mix_s1a0_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            recon_mix_s0a1_features, _, _  = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)
            
            layer_names = ['conv1_1', 'conv2_1',  'conv3_1',  'conv4_1', 'conv5_1'] #['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s1a0_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name], loss_type='l2') / (layer_weight*layer_weight * len(layer_names))*2
                #############################################################################################3
        g_loss = g_loss + G_loss_frecon

You can find it here:

Nonlinear_Face_3DMM/model_non_linear_3DMM_proxy.py

Line 445 in 662098a

if self.is_using_frecon:

@HOMGH Thank you very much!
I have another question. Do I need to modify the variables G_images_mix_s1a0_300W_vgg and G_images_mix_s0a1_300W_vgg in this code?

No, I don't think so. Actually I didn't get good results after training for 40 epochs which took me around 20 days! So I'm not sure whether this code works or not!

Hi, I would like to ask if the file you are running is model_non_linear.py or model_non_linear_proxy.py?
I am now going to try to run the model_non_linear.py file,so I want to ask whether other variables in the file proxy.py need to be modified or discarded. The following is the code I wrote:
`G_loss_frecon = tf.zeros(1)
if self.is_using_frecon:
from vgg_face import vgg_face

            G_images_300W_vgg = (tf.multiply(G_images_300W, G_images_300W_mask) + 1.0)*127.5
            # G_images_mix_s1a0_300W_vgg = (G_images_mix_s1a0_300W + 1.0) * 127.5
            # G_images_mix_s0a1_300W_vgg = (G_images_mix_s0a1_300W + 1.0) * 127.5
            input_images_300W_vgg = (input_images_300W + 1.0) * 127.5

            input_features, _, _ = vgg_face('vgg-face.mat', input_images_300W_vgg)
            recon_features, _, _ = vgg_face('vgg-face.mat', G_images_300W_vgg)
            # recon_mix_s1a0_features, _, _ = vgg_face('vgg-face.mat', G_images_mix_s1a0_300W_vgg)
            # recon_mix_s0a1_features, _, _ = vgg_face('vgg-face.mat', G_images_mix_s0a1_300W_vgg)

            layer_names = ['conv1_1', 'conv2_1', 'conv3_1', 'conv4_1',
                           'conv5_1']  # ['conv1_2', 'conv2_2',  'conv3_3',  'conv4_3', 'conv5_3']
            layer_weights = [50, 500, 2000, 3000, 250]

            for i in range(len(layer_names)):
                layer_name = layer_names[i]
                layer_weight = layer_weights[i]
                G_loss_frecon += norm_loss(input_features[layer_name], recon_features[layer_name],
                                           loss_type='l2') / (layer_weight * layer_weight * len(layer_names)) * 2
                # G_loss_frecon += norm_loss(input_features[layer_name], recon_mix_s0a1_features[layer_name],
                #                            loss_type='l2') / (layer_weight * layer_weight * len(layer_names)) * 2

        g_loss = g_loss + G_loss_frecon`

I am not sure if it is correct, but when I run it, the system prompts an error. Can you help me? thank you!

from nonlinear_face_3dmm.

model_non_linear_proxy.py

I ran model_non_linear_proxy.py!

from nonlinear_face_3dmm.

model_non_linear_proxy.py

I ran model_non_linear_proxy.py!

Have you not tried to run the model_non_linear.py file before?

from nonlinear_face_3dmm.