talkuhulk / realworld-stylegan2-encoder Goto Github PK

View Code? Open in Web Editor NEW

114.0 8.0 27.0 54.92 MB

Various applications based on Stylegan2 Style mixing that can be inference on cpu.

License: MIT License

Python 99.58% Shell 0.42%

openvino face-editing pytorch stylegan cartoonify art ukiyoe sketch

realworld-stylegan2-encoder's Introduction

RealWorld StyleGan2 Encoder

The demo of different style with age edit.

Description

Various applications based on Stylegan2 Style mixing that can be inference on cpu.

Citation

This code is heavily based on stylegan2-pytorch. Thanks rosinality so much to make his work available 🙏🙏🙏

Application

The demo of different style of psp-mobile-256p.

The demo of different style of psp-res50-256p.

The demo of different style of e4e-mobile-1024p.

The demo of different style of e4e-res50-1024p.

The demo of different style with age edit of e4e-res50-1024p

The demo of different style with gender edit of e4e-res50-1024p

The demo of different style with pose edit of e4e-res50-1024p

The demo of different style with smile edit of e4e-res50-1024p

Pretrained Models

I provide some of the model to test.

Google Drive	百度云	Description
torch weight	torch weight 密码:0mgk	cartoon of e4e-mbv3 1024p and psp-mbv3 256p
onnx weight	onnx weight 密码:inn8	stylegan2 onnx, cartoon of e4e-mbv3 1024p and psp-mbv3 256p
openvino weight	openvino weight 密码:759q	openvino cartoon of e4e-mbv3 1024p and psp-mbv3

Comparison

All test on MacOs 11.4 | 2.6 GHz Intel Core i7 | 32 GB 2667 MHz DDR4

pSp-mbv3-256p

Name	Time(s)
torch	1.9860
onnx	0.5869
openvino	0.4533

e4e-mbv3-1024p

Name	Time(s)
torch	4.8058
onnx	1.5155
openvino	0.8690

Test

torch

python scripts/test.py \
--ckpt ./best_model.pt \
--network psp \
--platform torch \
--align \
--images_path ./test_images

onnx

python scripts/test.py \
--ckpt ./cartoon_psp_mobile_256p.onnx \
--network psp \
--platform onnx \
--align \
--images_path ./test_images

openvino

python scripts/test.py \
--ckpt_encoder ./art_mobile_encoder_1024p \
--ckpt_decoder ./art_decoder_1024p \
--network e4e \
--platform openvino \
--align \
--images_path ./test_images \
--edit \
--edit_direction ./editings/smile.npy

Convert

torch2onnx

function can find in tools/torch2onnx.py

onnx2openvino

Inference Engine version: 2021.4.0-3839-cd81789d294-releases/2021/4 Model Optimizer version: 2021.4.0-3839-cd81789d294-releases/2021/4

python3 mo.py \
--input_model psp_mobile_256p.onnx \
--output_dir ./openvino  \
--data_type FP16 \
--mean_values [127.5,127.5,127.5] \
--scale_values [127.5,127.5,127.5] \
--move_to_preprocess

realworld-stylegan2-encoder's People

Contributors

Stargazers

Watchers

realworld-stylegan2-encoder's Issues

Convert stylegan2 to onnx

I tried to convert rosinality's stylegan2 to onnx but failed. There was unsupported operations. But it seems your implementation of stylegan can be sucessfully converted. What did you do to make this happen?

can you provide The demo of different style of e4e-mobile-1024p demo Pretrained Models.

I do not find the demo model (ckpt)in google Driver you provide? can you provide?
thank you very much

ValueError: setting an array element with a sequence............

when I'm tried to run model for Openvino I'm getting following error

-ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 3.

and also best_model is not defined for the first cases if anyone have then please share it to me

waiting for colab book of your code and also improve readme.md cause its not defining which model put in the which directory

Thanks.

ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 3.

having this unknown error.
i used sample images as well and new images as well. but nothing is solving this issue

python scripts/test.py --ckpt_encoder ./art_mobile_encoder_1024p --ckpt_decoder ./art_decoder_1024p --network e4e --platform openvino --align --images_path ./test_images --edit --edit_direction ./editings/smile.npy
stylegan2 generating...: 0it [00:00, ?it/s]/home/smartdev/realworld/./tools/common.py:86: DeprecationWarning: QUAD is deprecated and will be removed in Pillow 10 (2023-07-01). Use Transform.QUAD instead.
img = img.transform((transform_size, transform_size), Image.QUAD, (quad + 0.5).flatten(), Image.BILINEAR)
/home/smartdev/realworld/./tools/common.py:86: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.
img = img.transform((transform_size, transform_size), Image.QUAD, (quad + 0.5).flatten(), Image.BILINEAR)
stylegan2 generating...: 0it [00:01, ?it/s]
Traceback (most recent call last):
File "/home/smartdev/realworld/scripts/test.py", line 404, in
test_openvino(args)
File "/home/smartdev/realworld/scripts/test.py", line 172, in test_openvino
res = exec_decoder.infer(inputs={decoder_input_blob: [dlatents]})
File "ie_api.pyx", line 1080, in openvino.inference_engine.ie_api.ExecutableNetwork.infer
File "ie_api.pyx", line 1431, in openvino.inference_engine.ie_api.InferRequest.infer
File "ie_api.pyx", line 1452, in openvino.inference_engine.ie_api.InferRequest.infer
File "ie_api.pyx", line 1581, in openvino.inference_engine.ie_api.InferRequest._fill_inputs
ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 3.

RuntimeError: Unable to open /Users/hulk/Documents/InKe/scripts/align/shape_predictor_68_face_landmarks.dat

Hello, I am having the following problem with the torch framework. Can you provide this predictor please?

e4e decoder checkpoint mismatch

Hi,

While running the e4e model with torch platform, following mismatch error occurs

Unexpected key(s) in state_dict: "convs.12.conv.weight", "convs.12.conv.blur.kernel", "convs.12.conv.modulation.weight", "convs.12.conv.modulation.bias", "convs.12.noise.weight", "convs.12.activate.bias", "convs.13.conv.weight", "convs.13.conv.modulation.weight", "convs.13.conv.modulation.bias", "convs.13.noise.weight", "convs.13.activate.bias", "convs.14.conv.weight", "convs.14.conv.blur.kernel", "convs.14.conv.modulation.weight", "convs.14.conv.modulation.bias", "convs.14.noise.weight", "convs.14.activate.bias", "convs.15.conv.weight", "convs.15.conv.modulation.weight", "convs.15.conv.modulation.bias", "convs.15.noise.weight", "convs.15.activate.bias", "to_rgbs.6.bias", "to_rgbs.6.upsample.kernel", "to_rgbs.6.conv.weight", "to_rgbs.6.conv.modulation.weight", "to_rgbs.6.conv.modulation.bias", "to_rgbs.7.bias", "to_rgbs.7.upsample.kernel", "to_rgbs.7.conv.weight", "to_rgbs.7.conv.modulation.weight", "to_rgbs.7.conv.modulation.bias", "noises.noise_13", "noises.noise_14", "noises.noise_15", "noises.noise_16"

Either the parameters to construct decoder are wrong in the test script or the checkpoint was saved with different parameters.

Reconstruction Steps

Download model from Google drive
Run the following command

python scripts/test.py \
--ckpt ./torch/e4e_encode_mobile_cartoon.pt \
--network e4e \
--platform torch \
--align \
--images_path ./test-images

Can anyone verify it?

can you please provide a sample for e4e_encode_mobile_cartoon, need it to test on Indian faces.

Need the e4e mbv3 model without the age edit. Can I get it please or could you share the training procedure?

training code

great job, is the training code going to be released?