(base) PS C:\deepdream-test\DualStyleGAN> conda env create -f ./environment/dualstylegan_env.yaml
Collecting package metadata (repodata.json): done
Solving environment: failed

ResolvePackageNotFound:

• ca-certificates==2022.2.1=h06a4308_0
• libgcc-ng==9.3.0=h2828fa1_19
• faiss==1.7.1=py38h7b17aaf_0_cpu
• libedit==3.1.20191231=he28a2e2_2
• python-lmdb==1.2.1=py38h2531618_1
• matplotlib-base==3.3.4=py38h62a2d02_0
• libffi==3.2.1=he1b5a44_1007
• certifi==2021.10.8=py38h06a4308_2
• _libgcc_mutex==0.1=conda_forge
• scikit-image==0.18.1=py38ha9443f7_0
• libfaiss==1.7.1=hb573701_0_cpu
• python==3.8.3=cpython_he5300dc_0
• setuptools==49.6.0=py38h578d9bd_3
• pytorch==1.7.1=py3.8_cuda10.1.243_cudnn7.6.3_0
• libstdcxx-ng==9.3.0=h6de172a_19
• pillow==8.3.1=py38h2c7a002_0

Hi, I want to destylize the cartoon data, however, I can't fine-tune the stylegan due to the lack of GPU servers, could you provide the checkpoint "fintune-000600.pt"? Thanks.

Hi, would you be interested in adding DualStyleGAN to Hugging Face? The Hub offers free hosting, and it would make your work more accessible and visible to the rest of the ML community. We can create a username similar to github to add the models/spaces/datasets to.

Example from other organizations:
Keras: https://huggingface.co/keras-io
Microsoft: https://huggingface.co/microsoft
Facebook: https://huggingface.co/facebook

Example spaces with repos:
github: https://github.com/salesforce/BLIP
Spaces: https://huggingface.co/spaces/akhaliq/BLIP

github: https://github.com/facebookresearch/omnivore
Spaces: https://huggingface.co/spaces/akhaliq/omnivore

and here are guides for adding spaces/models/datasets to your org

How to add a Space: https://huggingface.co/blog/gradio-spaces
how to add models: https://huggingface.co/docs/hub/adding-a-model
uploading a dataset: https://huggingface.co/docs/datasets/upload_dataset.html

Please let us know if you would be interested and if you have any questions, we can also help with the technical implementation.

Thank you for your great work!

I have a question about the dataset split for the training and test. For example, the cartoon dataset has 317 images. To my understanding, these 317 images are used in the training and test at the same time. Did I understand correctly?

As shown in the picture above, the result of style_transfer.py does not come out properly. Why is that?

The output of the terminal is as follows

(dualstylegan_env) [lucass@nipa2019-0211 DualStyleGAN]$ python style_transfer.py
/home/lucass/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py:266: UserWarning:

!! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (c++) is not compatible with the compiler Pytorch was
built with for this platform, which is g++ on linux. Please
use g++ to to compile your extension. Alternatively, you may
compile PyTorch from source using c++, and then you can also use
c++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!! WARNING !!

warnings.warn(WRONG_COMPILER_WARNING.format(
/home/lucass/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py:266: UserWarning:

!! WARNING !!

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
Your compiler (c++) is not compatible with the compiler Pytorch was
built with for this platform, which is g++ on linux. Please
use g++ to to compile your extension. Alternatively, you may
compile PyTorch from source using c++, and then you can also use
c++ to compile your extension.

See https://github.com/pytorch/pytorch/blob/master/CONTRIBUTING.md for help
with compiling PyTorch from source.
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

!! WARNING !!

warnings.warn(WRONG_COMPILER_WARNING.format(
Load options
align_face: False
content: ./data/content/081680.jpg
data_path: ./data/
exstyle_name: exstyle_code.npy
model_name: generator.pt
model_path: ./checkpoint/
name: cartoon_transfer
output_path: ./output/
preserve_color: False
style: cartoon
style_id: 2
truncation: 0.75
weight: [0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

Loading pSp from checkpoint: ./checkpoint/encoder.pt
Load models successfully!
Generate images successfully!
Save images successfully!
(dualstylegan_env) [lucass@nipa2019-0211 DualStyleGAN]$

系统是ubuntu 20.04,用的是腾讯去的服务器，没有显卡，把device改为了cpu
运行不成功，请问什么原因

Load options
align_face: False
content: ./data/content/081680.jpg
data_path: ./data/
exstyle_name: refined_exstyle_code.npy
model_name: generator.pt
model_path: ./checkpoint/
name: cartoon_transfer
output_path: ./output/
preserve_color: False
style: cartoon
style_id: 53
truncation: 0.75
weight: [0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 0.75, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]

Loading pSp from checkpoint: ./checkpoint/encoder.pt
Killed

Hello,

I faced this problem when fine-tuning stylegen model. I am not sure what cause this error, either dualstylegan or stylegen, but it would be great if someone could have a look at it.

Traceback (most recent call last):
File "finetune_stylegan.py", line 14, in
from util import data_sampler, requires_grad, accumulate, sample_data, d_logistic_loss, d_r1_loss, g_nonsaturating_loss, g_path_regularize, make_noise, mixing_noise, set_grad_none
File "/home/donghyun/Desktop/training/DualStyleGAN/util.py", line 10, in
from model.stylegan.op import conv2d_gradfix
File "/home/donghyun/Desktop/training/DualStyleGAN/model/stylegan/op/init.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "/home/donghyun/Desktop/training/DualStyleGAN/model/stylegan/op/fused_act.py", line 11, in
fused = load(
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 969, in load
return _jit_compile(
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1196, in _jit_compile
return _import_module_from_library(name, build_directory, is_python_module)
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1547, in _import_module_from_library
return imp.load_module(module_name, file, path, description)
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/imp.py", line 242, in load_module
return load_dynamic(name, filename, file)
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/imp.py", line 342, in load_dynamic
return _load(spec)
ImportError: /home/donghyun/.cache/torch_extensions/fused/fused.so: undefined symbol: _ZN3c104cuda20getCurrentCUDAStreamEa
Traceback (most recent call last):
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/distributed/launch.py", line 260, in
main()
File "/home/donghyun/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/distributed/launch.py", line 255, in main
raise subprocess.CalledProcessError(returncode=process.returncode,
subprocess.CalledProcessError: Command '['/home/donghyun/anaconda3/envs/dualstylegan_env/bin/python', '-u', 'finetune_stylegan.py', '--local_rank=0', '--iter', '600', '--batch', '4', '--ckpt', './checkpoint/stylegan2-ffhq-config-f.pt', '--style', 'mystyle', '--augment', './data/mystyle/lmdb/']' returned non-zero exit status 1.

Hi, thanks for sharing code.
I am a newcomer in the field of style transfer. I follow the README.md, but I get bad result like this:

I want to know where the problem is.

Hoping for hearing from you soon.

I am experimenting DualStyleGAN with my custom dataset. I have 283 images and I trained StyleGAN for 600 iterations. Then, I started to destylize it. I set 300 iterations and 4 batch. I expected for it to end when it became 283/283 but now it continues.
[448/283] Lperc: 1.254; Lnoise: 0.000; LID: 0.100; Lreg: 0.315; lr: 0.000: 100% Also, I'm not sure if the destylization outputs are as they should be.

Need I write one myself？

Hi, thanks for sharing code.
I download the Anime dataset according to the README.md, but I get bad result like this:

Can you provide more details about trainning Anime dataset.
Here are some of my training results, is that correct？I used your provided pretrain model generator-pretrain.pt so I skip stage I&II, and fintune dualstylegan(stage III) using parameters as peper:


The picture above is fintune-004800.png，I use one GPU, so I trained 8*600=4800 with batch=4

The picture above is destylization log picture.

The picture above is dualstylegan-002000.jpg
Hoping for hearing from you soon.

i'm getting this error when building my dataset
it says i don't have enough space on my disk but i have 280gb available
i'm running on windows 10 by the way

c:\deepdream-test\DualStyleGAN>python ./model/stylegan/prepare_data.py --out ./data/psychedelic/lmdb/ --n_worker 1 --size 1024 ./data/psychedelic/images/
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Make dataset of image sizes: 1024
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\prepare_data.py", line 104, in <module>
    with lmdb.open(args.out, map_size=1024 ** 4, readahead=False) as env:
lmdb.Error: ./data/psychedelic/lmdb/: There is not enough space on the disk.

if i just run the second line of code i get this error

c:\deepdream-test\DualStyleGAN>python ./model/stylegan/prepare_data.py --out ./data/psychedelic/lmdb/ --n_worker 1 --size 1024 ./data/psychedelic/images/
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Make dataset of image sizes: 1024
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\prepare_data.py", line 104, in <module>
    with lmdb.open(args.out, map_size=1024 ** 4, readahead=False) as env:
lmdb.Error: ./data/psychedelic/lmdb/: There is not enough space on the disk.


c:\deepdream-test\DualStyleGAN>python ./model/stylegan/prepare_data.py --out ./data/psychedelic/lmdb/ --n_worker 1 --size 1024 ./data/psychedelic/images/
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Make dataset of image sizes: 1024
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\prepare_data.py", line 104, in <module>
    with lmdb.open(args.out, map_size=1024 ** 4, readahead=False) as env:
lmdb.Error: ./data/psychedelic/lmdb/: There is not enough space on the disk.


c:\deepdream-test\DualStyleGAN>python -m torch.distributed.launch --nproc_per_node=1 --master_port=8765 finetune_stylegan.py --iter 600 --batch 16 --ckpt ./checkpoint/stylegan2-ffhq-config-f.pt --style psychedelic --augment ./data/psychedelic/lmdb/
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
NOTE: Redirects are currently not supported in Windows or MacOs.
C:\python\lib\site-packages\torch\distributed\launch.py:163: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
  logger.warn(
The module torch.distributed.launch is deprecated and going to be removed in future.Migrate to torch.distributed.run
WARNING:torch.distributed.run:--use_env is deprecated and will be removed in future releases.
 Please read local_rank from `os.environ('LOCAL_RANK')` instead.
INFO:torch.distributed.launcher.api:Starting elastic_operator with launch configs:
  entrypoint       : finetune_stylegan.py
  min_nodes        : 1
  max_nodes        : 1
  nproc_per_node   : 1
  run_id           : none
  rdzv_backend     : static
  rdzv_endpoint    : 127.0.0.1:8765
  rdzv_configs     : {'rank': 0, 'timeout': 900}
  max_restarts     : 3
  monitor_interval : 5
  log_dir          : None
  metrics_cfg      : {}

INFO:torch.distributed.elastic.agent.server.local_elastic_agent:log directory set to: c:\AXIOM~2.FFM\ffmpeg\bin\_00CE~1\torchelastic_7xqlov8k\none_zvc0_8u4
INFO:torch.distributed.elastic.agent.server.api:[default] starting workers for entrypoint: python.exe
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
C:\python\lib\site-packages\torch\distributed\elastic\utils\store.py:52: FutureWarning: This is an experimental API and will be changed in future.
  warnings.warn(
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=0
  master_addr=127.0.0.1
  master_port=8765
  group_rank=0
  group_world_size=1
  local_ranks=[0]
  role_ranks=[0]
  global_ranks=[0]
  role_world_sizes=[1]
  global_world_sizes=[1]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: c:\AXIOM~2.FFM\ffmpeg\bin\_00CE~1\torchelastic_7xqlov8k\none_zvc0_8u4\attempt_0\0\error.json
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Load options
ada_every: 256
ada_length: 500000
ada_target: 0.6
augment: True
augment_p: 0
batch: 16
channel_multiplier: 2
ckpt: ./checkpoint/stylegan2-ffhq-config-f.pt
d_reg_every: 16
g_reg_every: 4
iter: 600
local_rank: 0
lr: 0.002
mixing: 0.9
model_path: ./checkpoint/
n_sample: 9
path: ./data/psychedelic/lmdb/
path_batch_shrink: 2
path_regularize: 2
r1: 10
save_every: 10000
size: 1024
style: psychedelic
wandb: False
**************************************************************************************************
load model: ./checkpoint/stylegan2-ffhq-config-f.pt
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\finetune_stylegan.py", line 380, in <module>
    dataset = MultiResolutionDataset(args.path, transform, args.size)
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\dataset.py", line 23, in __init__
    self.length = int(txn.get('length'.encode('utf-8')).decode('utf-8'))
AttributeError: 'NoneType' object has no attribute 'decode'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 14500) of binary: C:\python\python.exe
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 3/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=1
  master_addr=127.0.0.1
  master_port=8765
  group_rank=0
  group_world_size=1
  local_ranks=[0]
  role_ranks=[0]
  global_ranks=[0]
  role_world_sizes=[1]
  global_world_sizes=[1]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: c:\AXIOM~2.FFM\ffmpeg\bin\_00CE~1\torchelastic_7xqlov8k\none_zvc0_8u4\attempt_1\0\error.json
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Load options
ada_every: 256
ada_length: 500000
ada_target: 0.6
augment: True
augment_p: 0
batch: 16
channel_multiplier: 2
ckpt: ./checkpoint/stylegan2-ffhq-config-f.pt
d_reg_every: 16
g_reg_every: 4
iter: 600
local_rank: 0
lr: 0.002
mixing: 0.9
model_path: ./checkpoint/
n_sample: 9
path: ./data/psychedelic/lmdb/
path_batch_shrink: 2
path_regularize: 2
r1: 10
save_every: 10000
size: 1024
style: psychedelic
wandb: False
**************************************************************************************************
load model: ./checkpoint/stylegan2-ffhq-config-f.pt
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\finetune_stylegan.py", line 380, in <module>
    dataset = MultiResolutionDataset(args.path, transform, args.size)
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\dataset.py", line 23, in __init__
    self.length = int(txn.get('length'.encode('utf-8')).decode('utf-8'))
AttributeError: 'NoneType' object has no attribute 'decode'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 28776) of binary: C:\python\python.exe
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 2/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=2
  master_addr=127.0.0.1
  master_port=8765
  group_rank=0
  group_world_size=1
  local_ranks=[0]
  role_ranks=[0]
  global_ranks=[0]
  role_world_sizes=[1]
  global_world_sizes=[1]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: c:\AXIOM~2.FFM\ffmpeg\bin\_00CE~1\torchelastic_7xqlov8k\none_zvc0_8u4\attempt_2\0\error.json
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Load options
ada_every: 256
ada_length: 500000
ada_target: 0.6
augment: True
augment_p: 0
batch: 16
channel_multiplier: 2
ckpt: ./checkpoint/stylegan2-ffhq-config-f.pt
d_reg_every: 16
g_reg_every: 4
iter: 600
local_rank: 0
lr: 0.002
mixing: 0.9
model_path: ./checkpoint/
n_sample: 9
path: ./data/psychedelic/lmdb/
path_batch_shrink: 2
path_regularize: 2
r1: 10
save_every: 10000
size: 1024
style: psychedelic
wandb: False
**************************************************************************************************
load model: ./checkpoint/stylegan2-ffhq-config-f.pt
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\finetune_stylegan.py", line 380, in <module>
    dataset = MultiResolutionDataset(args.path, transform, args.size)
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\dataset.py", line 23, in __init__
    self.length = int(txn.get('length'.encode('utf-8')).decode('utf-8'))
AttributeError: 'NoneType' object has no attribute 'decode'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 24204) of binary: C:\python\python.exe
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:[default] Worker group FAILED. 1/3 attempts left; will restart worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Stopping worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous'ing worker group
INFO:torch.distributed.elastic.agent.server.api:[default] Rendezvous complete for workers. Result:
  restart_count=3
  master_addr=127.0.0.1
  master_port=8765
  group_rank=0
  group_world_size=1
  local_ranks=[0]
  role_ranks=[0]
  global_ranks=[0]
  role_world_sizes=[1]
  global_world_sizes=[1]

INFO:torch.distributed.elastic.agent.server.api:[default] Starting worker group
INFO:torch.distributed.elastic.multiprocessing:Setting worker0 reply file to: c:\AXIOM~2.FFM\ffmpeg\bin\_00CE~1\torchelastic_7xqlov8k\none_zvc0_8u4\attempt_3\0\error.json
C:\python\lib\site-packages\numpy\_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs:
C:\python\lib\site-packages\numpy\.libs\libopenblas.EL2C6PLE4ZYW3ECEVIV3OXXGRN2NRFM2.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.GK7GX5KEQ4F6UYO3P26ULGBQYHGQO7J4.gfortran-win_amd64.dll
C:\python\lib\site-packages\numpy\.libs\libopenblas.XWYDX2IKJW2NMTWSFYNGFUWKQU3LYTCZ.gfortran-win_amd64.dll
  warnings.warn("loaded more than 1 DLL from .libs:"
Load options
ada_every: 256
ada_length: 500000
ada_target: 0.6
augment: True
augment_p: 0
batch: 16
channel_multiplier: 2
ckpt: ./checkpoint/stylegan2-ffhq-config-f.pt
d_reg_every: 16
g_reg_every: 4
iter: 600
local_rank: 0
lr: 0.002
mixing: 0.9
model_path: ./checkpoint/
n_sample: 9
path: ./data/psychedelic/lmdb/
path_batch_shrink: 2
path_regularize: 2
r1: 10
save_every: 10000
size: 1024
style: psychedelic
wandb: False
**************************************************************************************************
load model: ./checkpoint/stylegan2-ffhq-config-f.pt
Traceback (most recent call last):
  File "c:\deepdream-test\DualStyleGAN\finetune_stylegan.py", line 380, in <module>
    dataset = MultiResolutionDataset(args.path, transform, args.size)
  File "c:\deepdream-test\DualStyleGAN\model\stylegan\dataset.py", line 23, in __init__
    self.length = int(txn.get('length'.encode('utf-8')).decode('utf-8'))
AttributeError: 'NoneType' object has no attribute 'decode'
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 12700) of binary: C:\python\python.exe
ERROR:torch.distributed.elastic.agent.server.local_elastic_agent:[default] Worker group failed
INFO:torch.distributed.elastic.agent.server.api:Local worker group finished (FAILED). Waiting 300 seconds for other agents to finish
C:\python\lib\site-packages\torch\distributed\elastic\utils\store.py:70: FutureWarning: This is an experimental API and will be changed in future.
  warnings.warn(
INFO:torch.distributed.elastic.agent.server.api:Done waiting for other agents. Elapsed: 0.0005078315734863281 seconds
{"name": "torchelastic.worker.status.FAILED", "source": "WORKER", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": 0, "group_rank": 0, "worker_id": "12700", "role": "default", "hostname": "adminoracle.home", "state": "FAILED", "total_run_time": 150, "rdzv_backend": "static", "raw_error": "{\"message\": \"<NONE>\"}", "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python.exe\", \"local_rank\": [0], \"role_rank\": [0], \"role_world_size\": [1]}", "agent_restarts": 3}}
{"name": "torchelastic.worker.status.SUCCEEDED", "source": "AGENT", "timestamp": 0, "metadata": {"run_id": "none", "global_rank": null, "group_rank": 0, "worker_id": null, "role": "default", "hostname": "adminoracle.home", "state": "SUCCEEDED", "total_run_time": 151, "rdzv_backend": "static", "raw_error": null, "metadata": "{\"group_world_size\": 1, \"entry_point\": \"python.exe\"}", "agent_restarts": 3}}
C:\python\lib\site-packages\torch\distributed\elastic\multiprocessing\errors\__init__.py:354: UserWarning:

**********************************************************************
               CHILD PROCESS FAILED WITH NO ERROR_FILE
**********************************************************************
CHILD PROCESS FAILED WITH NO ERROR_FILE
Child process 12700 (local_rank 0) FAILED (exitcode 1)
Error msg: Process failed with exitcode 1
Without writing an error file to <N/A>.
While this DOES NOT affect the correctness of your application,
no trace information about the error will be available for inspection.
Consider decorating your top level entrypoint function with
torch.distributed.elastic.multiprocessing.errors.record. Example:

  from torch.distributed.elastic.multiprocessing.errors import record

  @record
  def trainer_main(args):
     # do train
**********************************************************************
  warnings.warn(_no_error_file_warning_msg(rank, failure))
Traceback (most recent call last):
  File "C:\python\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\python\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\python\lib\site-packages\torch\distributed\launch.py", line 173, in <module>
    main()
  File "C:\python\lib\site-packages\torch\distributed\launch.py", line 169, in main
    run(args)
  File "C:\python\lib\site-packages\torch\distributed\run.py", line 621, in run
    elastic_launch(
  File "C:\python\lib\site-packages\torch\distributed\launcher\api.py", line 116, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "C:\python\lib\site-packages\torch\distributed\elastic\multiprocessing\errors\__init__.py", line 348, in wrapper
    return f(*args, **kwargs)
  File "C:\python\lib\site-packages\torch\distributed\launcher\api.py", line 245, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:
***************************************
      finetune_stylegan.py FAILED
=======================================
Root Cause:
[0]:
  time: 2022-04-07_23:20:42
  rank: 0 (local_rank: 0)
  exitcode: 1 (pid: 12700)
  error_file: <N/A>
  msg: "Process failed with exitcode 1"
=======================================
Other Failures:
  <NO_OTHER_FAILURES>
***************************************

1. Could you share how you modified pSp encoder with Z+?
   I could not find the details in the paper and codes.
   It seems you didn't use VAE encoder of the AgileGAN and also didn't inference Z mean and Z standard.
   The modification of pSp encoder is only passing output (Z+) of pSp encdoer to mapping layer indviually? Am I right?
2. How did you train the pSp encoder with Z+? Or you used the pretrained pSp encoder model directly?
   Looking forward to your reply.

Hi, While training all components of dual stylegan on my custom dataset I noticed that the script destylize.py like other scripts does not have the option to pass in custom paths for encoder and ckpt, also the output size of the gan. I did manage to change this according to my case but still had a doubt. My psp encoder was trained with a moco loss rather than ID loss since it is a non human domain. Should, I change that also to moco or will I obtain reasonable style codes with ID loss also.

Hi,
Thanks for putting out this work.
I am unable to train with my custom images dataset. The training is failing in the second step of Facial destylization : "Step 2: Fine-tune StyleGAN".

This is the same as an issue reported earlier by @dongyun-kim-arch.

I have given the error message below.

load model: ./checkpoint/stylegan2-ffhq-config-f.pt
0%| | 0/1 [00:00<?, ?it/s]/home/azureuser/DualStyleGAN_dataPrep/DualStyleGAN/model/stylegan/op/conv2d_gradfix.py:88: UserWarning: conv2d_gradfix not supported on PyTorch 1.12.1+cu102. Falling back to torch.nn.functional.conv2d().
warnings.warn(
0%| | 0/1 [00:00<?, ?it/s]
Traceback (most recent call last):
File "finetune_stylegan.py", line 391, in
train(args, loader, generator, discriminator, g_optim, d_optim, g_ema, device)
File "finetune_stylegan.py", line 159, in train
r1_loss = d_r1_loss(real_pred, real_img)
File "/home/azureuser/DualStyleGAN_dataPrep/DualStyleGAN/util.py", line 71, in d_r1_loss
grad_real, = autograd.grad(
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/autograd/init.py", line 276, in grad
return Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/autograd/function.py", line 253, in apply
return user_fn(self, *args)
File "/home/azureuser/DualStyleGAN_dataPrep/DualStyleGAN/model/stylegan/non_leaking.py", line 352, in backward
grad_input, grad_grid = GridSampleBackward.apply(grad_output, input, grid)
File "/home/azureuser/DualStyleGAN_dataPrep/DualStyleGAN/model/stylegan/non_leaking.py", line 361, in forward
grad_input, grad_grid = op(grad_output, input, grid, 0, 0, False)
TypeError: 'tuple' object is not callable
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 99134) of binary: /home/azureuser/anaconda3/envs/dualstylegan_env/bin/python3
Traceback (most recent call last):
File "/home/azureuser/anaconda3/envs/dualstylegan_env/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/home/azureuser/anaconda3/envs/dualstylegan_env/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 193, in
main()
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/distributed/run.py", line 752, in run
elastic_launch(
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/home/azureuser/.local/lib/python3.8/site-packages/torch/distributed/launcher/api.py", line 245, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

finetune_stylegan.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2022-11-11_12:08:03
host : asr-training.internal.cloudapp.net
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 99134)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Hi, @williamyang1991!

I'd like to use your method for anime faces. However, I guess that a Danbooru dataset is too huge...
So, would you provide the pre-processed dataset??

Hi, I am training my own dataset on Colab follwing the steps of Readme, but the training fails in the second step of Facial destylization : "Step 2: Fine-tune StyleGAN". The error information is as followed:

load model: ./checkpoint/stylegan2-ffhq-config-f.pt
0%| | 0/600 [00:00<?, ?it/s]
Traceback (most recent call last):
File "finetune_stylegan.py", line 391, in
train(args, loader, generator, discriminator, g_optim, d_optim, g_ema, device)
File "finetune_stylegan.py", line 115, in train
real_img = next(loader)
File "/content/DualStyleGAN/util.py", line 58, in sample_data
for batch in loader:
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 681, in next
data = self._next_data()
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/dataloader.py", line 721, in _next_data
data = self._dataset_fetcher.fetch(index) # may raise StopIteration
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in fetch
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/usr/local/lib/python3.7/dist-packages/torch/utils/data/_utils/fetch.py", line 49, in
data = [self.dataset[idx] for idx in possibly_batched_index]
File "/content/DualStyleGAN/model/stylegan/dataset.py", line 37, in getitem
img = Image.open(buffer)
File "/usr/local/lib/python3.7/dist-packages/PIL/Image.py", line 2657, in open
% (filename if filename else fp))
OSError: cannot identify image file <_io.BytesIO object at 0x7f070291c410>
ERROR:torch.distributed.elastic.multiprocessing.api:failed (exitcode: 1) local_rank: 0 (pid: 2003) of binary: /usr/bin/python3
Traceback (most recent call last):
File "/usr/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/usr/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 193, in
main()
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 189, in main
launch(args)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py", line 174, in launch
run(args)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/run.py", line 755, in run
)(*cmd_args)
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launcher/api.py", line 131, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/lib/python3.7/dist-packages/torch/distributed/launcher/api.py", line 247, in launch_agent
failures=result.failures,
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

finetune_stylegan.py FAILED

Failures:
<NO_OTHER_FAILURES>

Root Cause (first observed failure):
[0]:
time : 2022-11-23_07:27:01
host : a3b13d7b3fb3
rank : 0 (local_rank: 0)
exitcode : 1 (pid: 2003)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Hi! Amazing work!
I want to ask that do you mind share your pixar dataset?(not the youtube link...)

Hello! Can you tell me if there is some way to configure the inference so that it does not change the shape of the face as in the photo below?

Hi~ thanks for your amazing work. I try to follow your work but I am currently experiencing some problems.
Based on experience, I guess it should be an environmental problem.
My enviroment:
pytorch - 1.7.1
torchvision - 0.8.2
cuda - 10.1
cudnn - 7605
Hope to get your help. Many Thanks!

Traceback (most recent call last):
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1533, in _run_ninja_build
subprocess.run(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "style_transfer.py", line 4, in
from util import save_image, load_image
File "./DualStyleGAN-main/util.py", line 10, in
from model.stylegan.op import conv2d_gradfix
File "./DualStyleGAN-main/model/stylegan/op/init.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "./DualStyleGAN-main/model/stylegan/op/fused_act.py", line 11, in
fused = load(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 986, in load
return jit_compile(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1193, in jit_compile
write_ninja_file_and_build_library(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1297, in write_ninja_file_and_build_library
run_ninja_build(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1555, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'fused': [1/2] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/TH -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/THC -isystem ~/anaconda3/envs/dualstylegan_env/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c ~/DualStyleGAN-main/model/stylegan/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
FAILED: fused_bias_act_kernel.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/TH -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/THC -isystem ~/anaconda3/envs/dualstylegan_env/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c ~/DualStyleGAN-main/model/stylegan/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
nvcc fatal : Value 'c++14' is not defined for option 'std'
ninja: build stopped: subcommand failed.

Any clues on how to train/modify the input so that I can carry out inference on the entire body. I'm planning to use this on something like DCT-Net here

I have tried to resize the image to 1024/1024 but using a resized image gives some other results. Do I need to train it on different images or do I need to make changes to the code itself?

I noticed that your stylegan2-ffhq-config-f.pt is taken from rosinality.
Have you ever used the convert_weight.py in the rosinality?

I just used the convert_ weight. py to convert generator_yellow-stylegan2-config-f.pkl to .pt file, but there is a KeyError occurs when loading .pt. Can you tell me which key in the .pt corresponds to which key in the .pkl? Especially the 'g_ema'. Or something else was wrong?

Figure 1 is the BUG.
Figure 2 is your finetune_stylegan.py.
Figure 3 is the official stylegan2's run_training.py based on TensorFlow.

Hi, I find the loss function in Eq.(4) containing many parts. In styleGAN, only L_adv is used. I wonder if there is any motivation of using such a combination of loss. Is there any ablation study for IDloss? I only find ablation for reg_loss..

、

Hello!

I would like to train DualStyleGAN with my own data which is a particular style of human face datasets such as Pixar or Arcane.
I collected around 300 images and did all the preprocessing such as alignment and super-resolution.

Considering my domain (face) is similar to what you experimented with in your paper, I am not sure I could directly utilize your pertained model. For example, could I skip facial destylization and start with stage 3 in progressive fine-tuning? or should I start with step 3 in facial destylization?

It would be great if you could explain custom dataset training sequentially.
Thank you!

I have some questions about stage 1 & 2 of Pretrain DualStyleGAN on FFHQ.
Could you help me?

1. Could I use the pretrained model you provided (showing below) on several different dataset that I collected them myself.
   

2. I only find out stage 2 codes in pretrain_dualstylegan.py. Could you showd me codes of Stage 1 in pretrain_dualstylegan.py?

3. I am not understand stage1 very well from paper. In stage 1, you trained color transform blocks with style mixing FFHQ images?
   The purple hair in following figure you showed it on purpose. In most cases, it is regular hair color (black, yellow, ...) when we train model in stage1, right?
   

4. In stage 2, you mentioned you sampled z2~ from {z2,E(g(z2))}. Could I know why you do it this way instead of only z2 or E(g(z2)).
   

Looking forward to your reply

hi, there was something doubt for me that I had got only 4 face_images with a video more than 1h(109Mb). Could you please to give some suggestion?

inetune_dualstylegan.py pretrain_dualstylegan.py
(dualstylegan_env) ➜ DualStyleGAN git:(main) ✗ python style_transfer.py
/home/jp/miniconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/cuda/init.py:104: UserWarning:
NVIDIA GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
If you want to use the NVIDIA GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/

warnings.warn(incompatible_device_warn.format(device_name, capability, " ".join(arch_list), device_name))

Hello Researcher Yang.
Thank you for your excellent work!!! I followed your training pretty well and kept my style great! But the identity and structure change a lot, do you have any better solution for the difference in identity due to PSP inverse mapping? Or which part in between you need to fine tune according to the dataset, please give me some hints, thanks!

Hello, I am very interested in your job.
However, when I tried to use the model you provided, I found that the img_rec is very different from the original image. I don't know whether it is caused by the psp.encoder or the stylegan.generator.

Especially after using Webdemo, I found that the same input, Reconstructed on webdemo is very good.

So I have a huge confusion, do I need to retrain my psp.encoder to get a better instyle_code or do I need to do other work to solve the difficulties I am currently encountering.

@williamyang1991

Is there a way to compute the extrinsic style code for a given style image? The current code seems to load pre-computed style code of specific style images from .npy file.

Thank you for this amazing work!
I have a question about fine-tuning of the StyleGAN, which is the step 2 of Facial destylization.

While I reading your paper, I intuitively think that the residual path(ResBlock) are added in Generator. But, you actually added in the Discriminator. Why is that?

I found this error while running finetune_dualstylegan.py in the last step of learning.
I changed it to jpg because I thought the format of the picture might be a problem, but there must be a problem.
I would appreciate it if you could tell me how to solve it.

load model: /content/drive/MyDrive/generator-pretrain.pt
Loading pSp from checkpoint: /content/drive/MyDrive/encoder.pt
Loading ResNet ArcFace
Encoder model successfully loaded!
Traceback (most recent call last):
File "finetune_dualstylegan.py", line 529, in
Simg = transform2(Simg).unsqueeze(dim=3)
File "/usr/local/lib/python3.7/dist-packages/torchvision/transforms/transforms.py", line 67, in call
img = t(img)
File "/usr/local/lib/python3.7/dist-packages/torch/nn/modules/module.py", line 727, in call_impl
result = self.forward(*input, **kwargs)
File "/usr/local/lib/python3.7/dist-packages/torchvision/transforms/transforms.py", line 226, in forward
return F.normalize(tensor, self.mean, self.std, self.inplace)
File "/usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py", line 284, in normalize
tensor.sub(mean).div_(std)
RuntimeError: The size of tensor a (4) must match the size of tensor b (3) at non-singleton dimension 0

Hello. Is it possible to swap the encoder with some different encoder like ReStyle (https://github.com/yuval-alaluf/restyle-encoder) or Hyperstyle (https://github.com/yuval-alaluf/hyperstyle) for more accurate reconstruction of the face?

I followed your document to inference on cpu, but i found a small error

in DualStyleGAN/model/stylegan/op_cpu/conv2d_gradfix.py

it keep gets name error

you forgot to import contextlib!!

Check and Edit please

And thank you for your great work, I really enjoy using it

The model seems corrpted ? I have run the code above and no error occurred.

In Progressive Fine-Tuning Stage 1 & 2, you didn't use any style dataset or fine-tuned stylegan on style dataset, why does the result have kind of extrinsic style?

Hi, I followed the training step in Readme and apply the style_transfer.py on content face image but get distorted images, could you help me check the reason.

The script I run

python ./model/stylegan/prepare_data.py --out ./data/portrait/lmdb/ --n_worker 4 --size 1024 ./data/portrait/images/

python -m torch.distributed.launch --nproc_per_node=2 --master_port=8765 finetune_stylegan.py --iter 600 --batch 4 --ckpt ./checkpoint/stylegan2-ffhq-config-f.pt --style portrait --augment ./data/portrait/lmdb/

python destylize.py --model_name finetune-000600.pt --truncation 0.5 --batch 1 --iter 300 portrait

python -m torch.distributed.launch --nproc_per_node=2 --master_port=8765 finetune_dualstylegan.py --iter 1500 --batch 4 --ckpt ./checkpoint/generator-pretrain.pt --style_loss 0.25 --CX_loss 0.25 --perc_loss 0.75 --id_loss 0.75 --L2_reg_loss 0.02 --augment portrait

python refine_exstyle.py --lr_color 0.05 --lr_structure 0.1 --ckpt ./checkpoint/portrait/generator-001400.pt portrait

python style_transfer.py --style portrait --style_id 1 --model_name generator-001400.pt

this is the result image, and the third is the target style portrait, how can I fix the distortion or how to debug it?

And a second question, when I input my own face image but get a big change with respect to the expression and eyes, lips shape, how to skip this reconstruction stage and style-transforming my own face directly?

reconstruction image without align_face

reconstruction image with align_face

Hello! Thank you for your work!

I'm wondering how I can replace the encoder in the current implementation with https://github.com/eladrich/pixel2style2pixel encoder. First image "this implementation", Second image "pixel2style2pixel" implementation.

If I submit a latent vector from pixel2style2pixel, I get differentoutput.

But it is written in the README that it is possible to replace the encoder.

Thanks for your help!

Hello!

I am following the data preparation part to curate my own custom dataset, and I realized that face alignment and crop could be important. I saw additional files in the caricature and anime dataset such as txt file to pinpoint facial landmarks. It looks like the additional files were provided by other datasets, so I wonder if there are other tools or models I could utilize to get such additional information.

In this regard, I also wonder what preprocessing steps you used when testing the images created by Stable Diffusion.

Thank you!

Hi, can you provide the prompts for the three styles generated by StableDiffusion?
Thanks a lot!

I want slamdunk, Pixar, comic and arcane datasets, thank you very much!

Hi!

I suspect that users trying the demo will have a magic moment when they choose a style they love, combined with an image of themselves, and then generate a pleasing image as a result. With the current slider interface for style selection, this is not really possible.
I think providing a way for users to visually browse the styles and choose one they like would help chances of this tech becoming very popular and adopted across industry.
Fantastic work!

import argparse
import os
from argparse import Namespace

import numpy as np
import torch
import torchvision
from torch.nn import functional as F
from torchvision import transforms

from model.dualstylegan import DualStyleGAN
from model.encoder.psp import pSp
from util import save_image, load_image

from PIL import Image

class TestOptions():
def init(self):

    self.parser = argparse.ArgumentParser(description="Exemplar-Based Style Transfer")

    # Image path and file name to be transferred
    pic_path = './data/content/'
    pic_name = 'wowowo.jpeg'

    self.parser.add_argument("--content", type=str, default=pic_path + pic_name,
                             help="path of the content image")

    # Style setting / selection
    style_types = ['cartoon', 'caricature', 'anime', 'arcane', 'comic', 'pixar', 'slamdunk']
    # style_type : the default is 'cartoon',that's style_types[0]
    style_type = style_types[1]

    # style_id = [Cartoons]:(0-316)   [caricature]:(0-198)   [anime]:(0-173)
    #            [arcane]:(0-99)   [comic]:(0-100)   [pixar]:(0-121)   [slamdunk]:(0-119)
    # note : the value of style_id is an integer.
    #        For the portrait style comparison table, please refer to ./doc_images directory

    style_id = 52

    self.parser.add_argument("--style", type=str, default=style_type, help="target style type")

    self.parser.add_argument("--style_id", type=int, default=style_id, help="the id of the style image")


    self.parser.add_argument("--truncation", type=float, default=0.75,
                             help="truncation for intrinsic style code (content)")
    self.parser.add_argument("--weight", type=float, nargs=18, default=[0.75] * 7 + [1] * 11,
                             help="weight of the extrinsic style")

    # File header (including style name)
    self.parser.add_argument("--name", type=str, default=style_type+'_transfer',
                              help="filename to save the generated images")

    self.parser.add_argument("--preserve_color", action="store_true",
                             help="preserve the color of the content image")
    self.parser.add_argument("--model_path", type=str, default='./checkpoint/', help="path of the saved models")
    self.parser.add_argument("--model_name", type=str, default='generator.pt',
                             help="name of the saved dualstylegan")
    self.parser.add_argument("--output_path", type=str, default='./output/', help="path of the output images")
    self.parser.add_argument("--data_path", type=str, default='./data/', help="path of dataset")
    self.parser.add_argument("--align_face", action="store_true", help="apply face alignment to the content image")
    self.parser.add_argument("--exstyle_name", type=str, default=None, help="name of the extrinsic style codes")

    # python style_transfer.py --content ./data/content/unsplash-rDEOVtE7vOs.jpg --align_face --preserve_color \
    #  --style arcane --name arcane_transfer --style_id 13 \
    #  --weight 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 0.6 1 1 1 1 1 1 1

def parse(self):
    self.opt = self.parser.parse_args()
    if self.opt.exstyle_name is None:
        if os.path.exists(os.path.join(self.opt.model_path, self.opt.style, 'refined_exstyle_code.npy')):
            self.opt.exstyle_name = 'refined_exstyle_code.npy'
        else:
            self.opt.exstyle_name = 'exstyle_code.npy'
    args = vars(self.opt)
    print('Load options')
    for name, value in sorted(args.items()):
        print('%s: %s' % (str(name), str(value)))
    return self.opt

def run_alignment(args):
import dlib
from model.encoder.align_all_parallel import align_face
modelname = os.path.join(args.model_path, 'shape_predictor_68_face_landmarks.dat')
if not os.path.exists(modelname):
import wget, bz2
wget.download('http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2', modelname + '.bz2')
zipfile = bz2.BZ2File(modelname + '.bz2')
data = zipfile.read()
open(modelname, 'wb').write(data)
predictor = dlib.shape_predictor(modelname)
aligned_image = align_face(filepath=args.content, predictor=predictor)
return aligned_image

if name == "main":

# device = "cuda" # Change to CPU running  9-10-2022 
# At the same time, modify the 11 lines in the root directory util.py
# from model.stylegan.op import conv2d_gradfix
# To:
# from model.stylegan.op_cpu import conv2d_gradfix
#
# At the same time, modify the 11 lines in model.py in the model/stylegan directory
# from model.stylegan.op import FusedLeakyReLU, fused_leaky_relu, upfirdn2d, conv2d_gradfix
# To:
# from model.stylegan.op_cpu import FusedLeakyReLU,fused_leaky_relu, upfirdn2d, conv2d_gradfix

# device = "cuda"
device = "cpu"

parser = TestOptions()
args = parser.parse()
print('*' * 98)

# Determine the size of the input picture
# print(args.content)
# print(os.path.basename(args.content).split('.')[0] + '.' + os.path.basename(args.content).split('.')[1])

img = Image.open(args.content)
imgSize = img.size
w = img.width
h = img.height
f = img.format
if h != 1024:
    # print('Picture size:' + str(imgSize))
    # print('Picture size:Width ' + str(w), 'Height' + str(h), f)
    out_file = args.data_path + 'content/w_' + os.path.basename(args.content).split('.')[0] + '.' + os.path.basename(args.content).split('.')[1]
    # print(out_file)

    width = img.width
    height = img.height
    new_width = int(1024 * width / height)
    # print(new_width)
    out = img.resize((new_width, 1024), Image.Resampling.LANCZOS)
    out.save(out_file)
# else:
#   print('The picture is normal without modification')

transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize(mean=[0.5, 0.5, 0.5], std=[0.5, 0.5, 0.5]),
])

generator = DualStyleGAN(1024, 512, 8, 2, res_index=6)
generator.eval()

# ckpt = torch.load(os.path.join(args.model_path, args.style, args.model_name), map_location=lambda storage, loc: storage)
ckpt = torch.load(os.path.join(args.model_path, args.style, args.model_name), map_location=device)

generator.load_state_dict(ckpt["g_ema"])
generator = generator.to(device)

model_path = os.path.join(args.model_path, 'encoder.pt')
ckpt = torch.load(model_path, map_location=device)
opts = ckpt['opts']
opts['checkpoint_path'] = model_path
opts = Namespace(**opts)
opts.device = device
encoder = pSp(opts)
encoder.eval()
encoder.to(device)

exstyles = np.load(os.path.join(args.model_path, args.style, args.exstyle_name), allow_pickle='TRUE').item()

print('Load models successfully!')

with torch.no_grad():
    viz = []
    # load content image
    if args.align_face:
        I = transform(run_alignment(args)).unsqueeze(dim=0).to(device)
        I = F.adaptive_avg_pool2d(I, 1024)
    else:
        I = load_image(args.content).to(device)
    viz += [I]

    # reconstructed content image and its intrinsic style code
    img_rec, instyle = encoder(F.adaptive_avg_pool2d(I, 256), randomize_noise=False, return_latents=True,
                               z_plus_latent=True, return_z_plus_latent=True, resize=False)
    img_rec = torch.clamp(img_rec.detach(), -1, 1)
    viz += [img_rec]

    stylename = list(exstyles.keys())[args.style_id]
    latent = torch.tensor(exstyles[stylename]).to(device)
    if args.preserve_color:
        latent[:, 7:18] = instyle[:, 7:18]
    # extrinsic styte code
    exstyle = generator.generator.style(latent.reshape(latent.shape[0] * latent.shape[1], latent.shape[2])).reshape(
        latent.shape)

    # load style image if it exists
    S = None
    if os.path.exists(os.path.join(args.data_path, args.style, 'images/train', stylename)):
        S = load_image(os.path.join(args.data_path, args.style, 'images/train', stylename)).to(device)
        viz += [S]

    # style transfer 
    # input_is_latent: instyle is not in W space
    # z_plus_latent: instyle is in Z+ space
    # use_res: use extrinsic style path, or the style is not transferred
    # interp_weights: weight vector for style combination of two paths
    img_gen, _ = generator([instyle], exstyle, input_is_latent=False, z_plus_latent=True,
                           truncation=args.truncation, truncation_latent=0, use_res=True,
                           interp_weights=args.weight)
    img_gen = torch.clamp(img_gen.detach(), -1, 1)
    viz += [img_gen]

print('Generate images successfully!')


save_name = args.name + '_%d_%s' % (args.style_id, os.path.basename(args.content).split('.')[0])
# Generate preview effect picture
#save_image(torchvision.utils.make_grid(F.adaptive_avg_pool2d(torch.cat(viz, dim=0), 256), 4, 2).cpu(),
#           os.path.join(args.output_path, save_name + '_overview.jpg'))
# Generate final effect picture
save_image(img_gen[0].cpu(), os.path.join(args.output_path, save_name + '.jpg'))

print('Save images successfully!')

if os.path.exists(out_file):
    os.remove(out_file)

When I finetune DualStyleGAN, when d_r1_loss(real_pred, real_img), it call 'upfirdn2d.py', and RunError:

File "/home/tongtong/python_project/DualStyleGAN/model/stylegan/op/upfirdn2d.py", line 143, in backward
ctx.out_size,
File "/home/tongtong/python_project/DualStyleGAN/model/stylegan/op/upfirdn2d.py", line 42, in forward
g_pad_y1,
RuntimeError: input must be contiguous

I have tried to add .contiguous() after the input, but it doesn't work. What should I do?
Thanks in advance.

Hi~ thanks for your amazing work. I try to follow your work but I am currently experiencing some problems.
Based on experience, I guess it should be an environmental problem.
My enviroment:
pytorch - 1.3.1
torchvision - 1.7.1
cuda - 10.1
cudnn - 7605
Hope to get your help. Many Thanks!

Traceback (most recent call last):
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1533, in _run_ninja_build
subprocess.run(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/subprocess.py", line 512, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['ninja', '-v']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "style_transfer.py", line 4, in
from util import save_image, load_image
File "./DualStyleGAN-main/util.py", line 10, in
from model.stylegan.op import conv2d_gradfix
File "./DualStyleGAN-main/model/stylegan/op/init.py", line 1, in
from .fused_act import FusedLeakyReLU, fused_leaky_relu
File "./DualStyleGAN-main/model/stylegan/op/fused_act.py", line 11, in
fused = load(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 986, in load
return _jit_compile(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1193, in jit_compile
write_ninja_file_and_build_library(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1297, in _write_ninja_file_and_build_library
_run_ninja_build(
File "/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/utils/cpp_extension.py", line 1555, in run_ninja_build
raise RuntimeError(message) from e
RuntimeError: Error building extension 'fused': [1/2] /usr/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/TH -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/THC -isystem ~/anaconda3/envs/dualstylegan_env/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS -D__CUDA_NO_HALF2_OPERATORS --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c ~/DualStyleGAN-main/model/stylegan/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
FAILED: fused_bias_act_kernel.cuda.o
/usr/bin/nvcc -DTORCH_EXTENSION_NAME=fused -DTORCH_API_INCLUDE_EXTENSION_H -DPYBIND11_COMPILER_TYPE="gcc" -DPYBIND11_STDLIB="libstdcpp" -DPYBIND11_BUILD_ABI="cxxabi1011" -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/torch/csrc/api/include -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/TH -isystem ~/anaconda3/envs/dualstylegan_env/lib/python3.8/site-packages/torch/include/THC -isystem ~/anaconda3/envs/dualstylegan_env/include/python3.8 -D_GLIBCXX_USE_CXX11_ABI=0 -D__CUDA_NO_HALF_OPERATORS -D__CUDA_NO_HALF_CONVERSIONS_ -D__CUDA_NO_HALF2_OPERATORS__ --expt-relaxed-constexpr -gencode=arch=compute_61,code=sm_61 --compiler-options '-fPIC' -std=c++14 -c ~/DualStyleGAN-main/model/stylegan/op/fused_bias_act_kernel.cu -o fused_bias_act_kernel.cuda.o
nvcc fatal : Value 'c++14' is not defined for option 'std'
ninja: build stopped: subcommand failed.