mike9251 / simswap-inference-pytorch Goto Github PK

Unofficial Pytorch implementation (inference only) of the SimSwap: An Efficient Framework For High Fidelity Face Swapping

Python 100.00%

deepfake pytorch gfpgan simswap

simswap-inference-pytorch's Introduction

Unofficial Pytorch implementation (inference only) of the SimSwap: An Efficient Framework For High Fidelity Face Swapping

Updates

improved performance (up to 40% in some scenarios, it depends on frame resolution and number of swaps per frame).
fixed a problem with overlapped areas from close faces (#21)
added support for using GFPGAN model as an additional post-processing step to improve final image quality
added a toy gui app. Might be useful to understand how different pipeline settings affect output

Attention

This project is for technical and academic use only. Please do not apply it to illegal and unethical scenarios.

In the event of violation of the legal and ethical requirements of the user's country or region, this code repository is exempt from liability.

Preparation

Installation

# clone project
git clone https://github.com/mike9251/simswap-inference-pytorch
cd simswap-inference-pytorch

# [OPTIONAL] create conda environment
conda create -n myenv python=3.9
conda activate myenv

# install pytorch and torchvision according to instructions
# https://pytorch.org/get-started/

# install requirements
pip install -r requirements.txt

Important

Face detection will be performed on CPU. To run it on GPU you need to install onnx gpu runtime:

pip install onnxruntime-gpu==1.11.1

and modify one line of code in ...Anaconda3\envs\myenv\Lib\site-packages\insightface\model_zoo\model_zoo.py

Here, instead of passing None as the second argument to the onnx inference session

class ModelRouter:
    def __init__(self, onnx_file):
        self.onnx_file = onnx_file

    def get_model(self):
        session = onnxruntime.InferenceSession(self.onnx_file, None)
        input_cfg = session.get_inputs()[0]

pass a list of providers

class ModelRouter:
    def __init__(self, onnx_file):
        self.onnx_file = onnx_file

    def get_model(self):
        session = onnxruntime.InferenceSession(self.onnx_file, providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
        input_cfg = session.get_inputs()[0]

Otherwise simply use CPU onnx runtime with only a minor performance drop.

Weights

Weights for all models get downloaded automatically.

You can also download weights manually and put inside weights folder:

weights/face_detector_scrfd_10g_bnkps.onnx
weights/arcface_net.jit
weights/79999_iter.pth
weights/simswap_224_latest_net_G.pth - official 224x224 model
weights/simswap_512_390000_net_G.pth - unofficial 512x512 model (I took it here).
weights/GFPGANv1.4_ema.pth
weights/blend_module.jit

Inference

Web App

streamlit run app_web.py

Command line App

This repository supports inference in several modes, which can be easily configured with config files in the configs folder.

replace all faces on a target image / folder with images

python app.py --config-name=run_image.yaml

replace all faces on a video

python app.py --config-name=run_video.yaml

replace a specific face on a target image / folder with images

python app.py --config-name=run_image_specific.yaml

replace a specific face on a video

python app.py --config-name=run_video_specific.yaml

Config files contain two main parts:

data
- id_image - source image, identity of this person will be transferred.
- att_image - target image, attributes of the person on this image will be mixed with the person's identity from the source image. Here you can also specify a folder with multiple images - identity translation will be applied to all images in the folder.
- specific_id_image - a specific person on the att_image you would like to replace, leaving others untouched (if there's any other person).
- att_video - the same as att_image
- clean_work_dir - whether remove temp folder with images or not (for video configs only).
pipeline
- face_detector_weights - path to the weights file OR an empty string ("") for automatic weights downloading.
- face_id_weights - path to the weights file OR an empty string ("") for automatic weights downloading.
- parsing_model_weights - path to the weights file OR an empty string ("") for automatic weights downloading.
- simswap_weights - path to the weights file OR an empty string ("") for automatic weights downloading.
- gfpgan_weights - path to the weights file OR an empty string ("") for automatic weights downloading.
- device - whether you want to run the application using GPU or CPU.
- crop_size - size of images SimSwap models works with.
- checkpoint_type - the official model works with 224x224 crops and has different pre/post processings (imagenet like). Latest official repository allows you to train your own models, but the architecture and pre/post processings are slightly different (1. removed Tanh from the last layer; 2. normalization to [0...1] range). If you run the official 224x224 model then set this parameter to "official_224", otherwise "none".
- face_alignment_type - affects reference face key points coordinates. Possible values are "ffhq" and "none". Try both of them to see which one works better for your data.
- smooth_mask_kernel_size - a non-zero value. It's used for the post-processing mask size attenuation. You might want to play with this parameter.
- smooth_mask_iter - a non-zero value. The number of times a face mask is smoothed.
- smooth_mask_threshold - controls the face mask saturation. Valid values are in range [0.0...1.0]. Tune this parameter if there are artifacts around swapped faces.
- face_detector_threshold - values in range [0.0...1.0]. Higher value reduces probability of FP detections but increases the probability of FN.
- specific_latent_match_threshold - values in range [0.0...inf]. Usually takes small values around 0.05.
- enhance_output - whether to apply GFPGAN model or not as a post-processing step.

Overriding parameters with CMD

Every parameter in a config file can be overridden by specifying it directly with CMD. For example:

python app.py --config-name=run_image.yaml data.specific_id_image="path/to/the/image" pipeline.erosion_kernel_size=20

Video

Official 224x224 model, face alignment "none"

Official 224x224 model, face alignment "ffhq"

Unofficial 512x512 model, face alignment "none"

Unofficial 512x512 model, face alignment "ffhq"

License

For academic and non-commercial use only.The whole project is under the CC-BY-NC 4.0 license. See LICENSE for additional details.

Acknowledgements

simswap-inference-pytorch's People

Contributors

Stargazers

Watchers

simswap-inference-pytorch's Issues

OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.

(myenv) c:_N\simswap-inference-pytorch>streamlit run app_web.py

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501
Network URL: http://192.168.1.70:8501

OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/

Square around swapped face

Hi, after the GFPGAN update i'm getting squares around the swapped faces with and without using GFPGAN
Usually when the subject is not in front of the camera

The allignment type, erode mask value, smooth mask value and detector threshold are the same from the last version

Testing the same input with the older version does not make the squares appear

Faceshifter model?

Hey, anyone know if there's a way to use the sberswap pretrained model?

cloud gpu

hello nice to see this work,
i got issue when im using cloud gpu like colab or saturn.
how to change model zoo if im using colab or cloud gpu ?
when im start to streamlit on colab, im choose external url but nothing happen, just tab error.
please help, thanks

ERROR: No matching distribution found for onnxruntime==1.11.1

$ pip install -r requirements.txt
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting hydra-core>=1.1.0
Downloading hydra_core-1.3.1-py3-none-any.whl (154 kB)
-------------------------------------- 154.1/154.1 kB 3.1 MB/s eta 0:00:00
Collecting insightface==0.2.1
Downloading insightface-0.2.1-py2.py3-none-any.whl (24 kB)
Collecting kornia==0.6.5
Downloading kornia-0.6.5-py2.py3-none-any.whl (512 kB)
-------------------------------------- 512.8/512.8 kB 8.0 MB/s eta 0:00:00
Collecting moviepy==1.0.3
Downloading moviepy-1.0.3.tar.gz (388 kB)
------------------------------------- 388.3/388.3 kB 23.6 MB/s eta 0:00:00
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Collecting onnx==1.12.0
Downloading onnx-1.12.0-cp310-cp310-win_amd64.whl (11.5 MB)
--------------------------------------- 11.5/11.5 MB 18.7 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement onnxruntime==1.11.1 (from versions: 1.12.0, 1.12.1, 1.13.1)
ERROR: No matching distribution found for onnxruntime==1.11.1

RuntimeError: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 0

Video: https://www.dropbox.com/s/r331tr62dolpcop/Djokovic.mp4?dl=0
Frames: https://www.dropbox.com/s/mr15w1cecjdvqvg/frames.7z?dl=0

Tested on Colab.

When running the command !python app.py --config-name=run_video.yaml with the following config file parameters:

data:
  id_image: "${hydra:runtime.cwd}/demo_file/Iron_man.jpg"
  att_image: "none"
  specific_id_image: "none"
  att_video: "${hydra:runtime.cwd}/demo_file/Djokovic.mp4"
  output_dir: ${hydra:runtime.cwd}/output

pipeline:
  face_detector_weights: "${hydra:runtime.cwd}/weights/antelope/scrfd_10g_bnkps.onnx"
  face_id_weights: "${hydra:runtime.cwd}/weights/arcface_model/arcface_net.jit"
  parsing_model_weights: "${hydra:runtime.cwd}/weights/parsing_model/79999_iter.pth"
  simswap_weights: "${hydra:runtime.cwd}/weights/simswap_512/390000_net_G.pth"
  device: "cuda"
  crop_size: 512
  # it seems that the official 224 checkpoint works better with 'none' face alignment type
  checkpoint_type: "none" #"none
  face_alignment_type: "none" #"ffhq"
  erosion_kernel_size: 4

defaults:
  - _self_
  - override hydra/hydra_logging: disabled
  - override hydra/job_logging: disabled

hydra:
  output_subdir: null
  run:
    dir: .

I get this error after 47 frames:

10% 48/462 [00:18<02:38,  2.62it/s]
Error executing job with overrides: []
Traceback (most recent call last):
  File "app.py", line 72, in main
    app.run()
  File "app.py", line 62, in run
    output = self.model(att_img)
  File "/content/simswap-inference-pytorch/src/simswap.py", line 176, in __call__
    swapped_img = swapped_img * soft_face_mask + align_att_img_batch * (1 - soft_face_mask)
RuntimeError: The size of tensor a (3) must match the size of tensor b (2) at non-singleton dimension 0

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Last frames: 45,46,47
https://imgur.com/a/JR78c14

Possibility for adding GFPGAN ?

like here?
neuralchen/SimSwap#259

For improve gpu utilization

Are there any plans to add feature of batch processing

AssertionError: Torch not compiled with CUDA enabled

Regardless of which version of PyTorch is installed, this error message is encountered. It doesn't appear to be possible to run this without CUDA-compiled Torch. If, instead, I install Torch with conda and cudatoolkit=11.6, then the problem is that it cannot run because I have CUDA 11.7, and PyTorch is not yet compatible with that version of CUDA. There is an NVIDIA driver / CUDA mismatch. So I'm stuck having to run on CPU.

Please be so kind as to advise where your code should be tweaked to force CPU mode. Thank you.

Moviepy with nvenc

Torch not compiled with CUDA enabled

(sink2) C:_N\simswap-inference-pytorch-main>python app.py --config-name=run_image.yaml
Error executing job with overrides: []
Traceback (most recent call last):
File "C:_N\simswap-inference-pytorch-main\app.py", line 74, in
main()
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra\main.py", line 94, in decorated_main
_run_hydra(
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra_internal\utils.py", line 394, in _run_hydra
_run_app(
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra_internal\utils.py", line 457, in _run_app
run_and_report(
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra_internal\utils.py", line 223, in run_and_report
raise ex
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra_internal\utils.py", line 220, in run_and_report
return func()
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra_internal\utils.py", line 458, in
lambda: hydra.run(
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra_internal\hydra.py", line 132, in run
_ = ret.return_value
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra\core\utils.py", line 260, in return_value
raise self._return_value
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\hydra\core\utils.py", line 186, in run_job
ret.return_value = task_function(task_cfg)
File "C:_N\simswap-inference-pytorch-main\app.py", line 68, in main
app = Application(config)
File "C:_N\simswap-inference-pytorch-main\app.py", line 49, in init
self.model = SimSwap(
File "C:_N\simswap-inference-pytorch-main\src\simswap.py", line 42, in init
self.set_parameters(config)
File "C:_N\simswap-inference-pytorch-main\src\simswap.py", line 118, in set_parameters
self.set_smooth_mask_kernel_size(config.smooth_mask_kernel_size)
File "C:_N\simswap-inference-pytorch-main\src\simswap.py", line 169, in set_smooth_mask_kernel_size
self.re_initialize_soft_mask()
File "C:_N\simswap-inference-pytorch-main\src\simswap.py", line 160, in re_initialize_soft_mask
self.smooth_mask = SoftErosion(kernel_size=self.smooth_mask_kernel_size,
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\torch\nn\modules\module.py", line 989, in to
return self._apply(convert)
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\torch\nn\modules\module.py", line 688, in _apply
self.buffers[key] = fn(buf)
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\torch\nn\modules\module.py", line 987, in convert
return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
File "C:_N\Anaconda3\envs\sink2\lib\site-packages\torch\cuda_init.py", line 221, in _lazy_init
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled

(sink2) C:_N\simswap-inference-pytorch-main>

ашду onnxruntime_pybind11_state.pyd addresses the certain local files:

D:\a_work\1\s\onnxruntime\python\onnxruntime

tried with cuda:

(myenv) c:_N\simswap-inference-pytorch>streamlit run app_web.py

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8501
Network URL: http://192.168.1.70:8501

A new version of Streamlit is available.

See what's new at https://discuss.streamlit.io/c/announcements

Enter the following command to upgrade:
$ pip install streamlit --upgrade

2023-01-09 04:05:13.8153921 [E:onnxruntime:Default, provider_bridge_ort.cc:1022 onnxruntime::ProviderLibrary::Get] LoadLibrary failed with error 126 "" when trying to load "C:_N\Anaconda3\envs\myenv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
2023-01-09 04:05:13.8601772 [E:onnxruntime:Default, provider_bridge_ort.cc:1022 onnxruntime::ProviderLibrary::Get] LoadLibrary failed with error 126 "" when trying to load "C:_N\Anaconda3\envs\myenv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
2023-01-09 04:05:13.8770375 [E:onnxruntime:Default, provider_bridge_ort.cc:1022 onnxruntime::ProviderLibrary::Get] LoadLibrary failed with error 126 "" when trying to load "C:_N\Anaconda3\envs\myenv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
2023-01-09 04:05:13.867 Uncaught app exception
Traceback (most recent call last):
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 589, in get_or_create_cached_value
return_value = _read_from_cache(
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 349, in _read_from_cache
raise e
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 334, in _read_from_cache
return _read_from_mem_cache(
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 252, in _read_from_mem_cache
raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.runtime.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 565, in _run_script
exec(code, module.dict)
File "C:_N\simswap-inference-pytorch\app_web.py", line 157, in
model = load_model(config)
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 623, in wrapped_func
return get_or_create_cached_value()
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 607, in get_or_create_cached_value
return_value = non_optional_func(*args, **kwargs)
File "C:_N\simswap-inference-pytorch\app_web.py", line 111, in load_model
return SimSwap(
File "c:_N\simswap-inference-pytorch\src\simswap.py", line 64, in init
self.face_detector = get_model(
File "c:_N\simswap-inference-pytorch\src\model_loader.py", line 97, in get_model
model = models[model_name].model(**kwargs)
File "c:_N\simswap-inference-pytorch\src\FaceDetector\face_detector.py", line 26, in init
self.handler = model_zoo.get_model(str(model_path))
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\insightface\model_zoo\model_zoo.py", line 56, in get_model
model = router.get_model()
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\insightface\model_zoo\model_zoo.py", line 23, in get_model
session = onnxruntime.InferenceSession(self.onnx_file, providers=['CUDAExecutionProvider', 'CPUExecutionProvider'])
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 335, in init
self._create_inference_session(providers, provider_options, disabled_optimizers)
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 381, in _create_inference_session
sess.initialize_session(providers, provider_options, disabled_optimizers)
RuntimeError: D:\a_work\1\s\onnxruntime\python\onnxruntime_pybind_state.cc:548 onnxruntime::python::CreateExecutionProviderInstance CUDA_PATH is set but CUDA wasn't able to be loaded. Please install the correct version of CUDA and cuDNN as mentioned in the GPU requirements page (https://onnxruntime.ai/docs/reference/execution-providers/CUDA-ExecutionProvider.html#requirements), make sure they're in the PATH, and that your GPU is supported.

2023-01-09 04:05:13.885 Uncaught app exception
Traceback (most recent call last):
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 589, in get_or_create_cached_value
return_value = _read_from_cache(
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 349, in _read_from_cache
raise e
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 334, in _read_from_cache
return _read_from_mem_cache(
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 252, in _read_from_mem_cache
raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.runtime.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

2023-01-09 04:05:14.9111703 [E:onnxruntime:Default, provider_bridge_ort.cc:1022 onnxruntime::ProviderLibrary::Get] LoadLibrary failed with error 126 "" when trying to load "C:_N\Anaconda3\envs\myenv\lib\site-packages\onnxruntime\capi\onnxruntime_providers_cuda.dll"
2023-01-09 04:05:14.919 Uncaught app exception
Traceback (most recent call last):
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 589, in get_or_create_cached_value
return_value = _read_from_cache(
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 349, in _read_from_cache
raise e
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 334, in _read_from_cache
return _read_from_mem_cache(
File "C:_N\Anaconda3\envs\myenv\lib\site-packages\streamlit\runtime\legacy_caching\caching.py", line 252, in _read_from_mem_cache
raise CacheKeyNotFoundError("Key not found in mem cache")
streamlit.runtime.legacy_caching.caching.CacheKeyNotFoundError: Key not found in mem cache

During handling of the above exception, another exception occurred:

Video frames added from other video

Hey
When I use the faceswap video (single image) config and take a video that is, lets say 300 frames. The right after, if I run the config with a video of 200 frames, the 100 extra frames from before will be added to the video.

I simply fix it by deleting the folder with all the frames from last time, so this could probably be added to the code as well :)
I know simswap just deletes them every time it starts a run.

如何提高推理速度？我尝试加入多线程但是不生效

Low GPU usage - 3it/s on 3080ti GPU

Hi,
I'm getting 3 it/s with onnxruntime-gpu 1.11.1, is this a normal inference speed while using GPU?
15% GPU utilization and 50% CPU

Always error when video finishing compile image, Hydra.

when finishing image compile to video, always got this error.

`
Error executing job with overrides: []
Traceback (most recent call last):
File "/ironman/simswap-inference-pytorch/app.py", line 70, in main
app.run()
File "/ironman/simswap-inference-pytorch/app.py", line 60, in run
output = self.model(att_img)
File "/ironman/simswap-inference-pytorch/src/simswap.py", line 230, in call
align_att_imgs, att_transforms, att_detection_score = self.run_detect_align(
File "/ironman/simswap-inference-pytorch/src/simswap.py", line 186, in run_detect_align
detection: Detection = self.face_detector(image)
File "/ironman/simswap-inference-pytorch/src/FaceDetector/face_detector.py", line 31, in call
bboxes, kpss = self.handler.detect(
File "/usr/local/lib/python3.9/dist-packages/insightface/model_zoo/scrfd.py", line 204, in detect
im_ratio = float(img.shape[0]) / img.shape[1]
AttributeError: 'NoneType' object has no attribute 'shape'

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
`

but when i change to another video, sometimes its work. please fix this.

Error executing job with overrides:

Hello,

I'm new to coding and machine learning and I've been having some trouble with the run_video config.
I've been trying to run it on certain videos but after about a quarter of the frames are swapped, I end up getting the following error in the cmd line and the program stops running:

Error executing job with overrides: []
Traceback (most recent call last):
File "C:\Simswap-inference-pytorch\simswap-inference-pytorch\app.py", line 70, in main
app.run()
File "C:\Simswap-inference-pytorch\simswap-inference-pytorch\app.py", line 60, in run
output = self.model(att_img)
File "C:\Simswap-inference-pytorch\simswap-inference-pytorch\src\simswap.py", line 323, in call
target_image = kornia.geometry.transform.warp_affine(
File "C:\Users\user\anaconda3\envs\simswap\lib\site-packages\kornia\geometry\transform\imgwarp.py", line 203, in warp_affine
grid = F.affine_grid(src_norm_trans_dst_norm[:, :2, :], [B, C, dsize[0], dsize[1]], align_corners=align_corners)
File "C:\Users\user\anaconda3\envs\simswap\lib\site-packages\torch\nn\functional.py", line 4320, in affine_grid
return torch.affine_grid_generator(theta, size, align_corners)
RuntimeError: Expected size for first two dimensions of batch2 tensor to be: [2, 3] but got: [1, 3].

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

Feature: GIF support

Hi, could you please implement GIF rendering? If I use a GIF as att_video, MoviePy cannot render:
ValueError: MoviePy couldn't find the codec associated with the filename. Provide the 'codec' parameter in write_videofile.
(Frames are generated correctly though.)

Only cpu!

does it able to run only on the CPU??????
I have no GPUs,
after installing Pytorch on the CPU it simply asks for GPU or CUDA.
@mike9251

Error in latest source

I'm receving this error after updating the repo. Very odd, it actually ran a few times with some videos before doing this.

Feature: Changing output name

Hey again (again)
When I'm batching videoswapping I used to be able to set the name of the swaps in the commandline. I can't find a way to do it right now and it helps alot when trying different settings and batching the videos.

So this is more a wish for a feature to be able to change the name of the output file :)

GFPGAN Model and multispecific

Hi :)

I have 2 questions please :

1: I wanted to know if its possible to change the gfpgan model for post processing ? Like using the 1.2 or 1.3 version ?

2: Is it possible to do a multispecific swap like in the original simswap ?

Thanks again for your work and time

Have a nice day

I'm having a problem with overlapping faces.

I guess to fix this I have to merge all the found faces separately. Can you help with this?

By the way, the app already finds softmasks of faces. After all, isn't it better to use them when merging faces into the picture?

Help

Which file do I have to modify to remove the deepfake watermark or how do I do it? I'm trying to introduce the deepfake topic to my students and don't want to blow the surprise by having the watermark. Thank you.

Please dm me on telegram if you know how @keronikateka

GPU not used in windows 11

Hi,

I managed to get this work great on Windows 10, RTX 3080 used at 99%.

I have updated my system to windows 11, installed everything the same way that I did on windows 10, but now my GPU is not fully used, this only using my CPU (CPU 80% GPU 13%)

I dont understand what I did wrong, is there something particular with windows 11 ?

Thanks a lot for your help.

I haven't been able to match the faces in the Multispecific folder.

I couldn't understand how I could use the multispecific feature available in the original SimSwap examples in your adaptation. Do we need to do anything special for this?

How to do a batch inference?

For example, the size of image here (It is not directing correctly. Should be on line 186, https://github.com/mike9251/simswap-inference-pytorch/blob/main/src/simswap.py#:~:text=detection%3A%20Detection%20%3D%20self.face_detector(image) ) is [4,3,256,256], how could we adapt the code?

Thank you in advance.

pipeline devices

Hello, my server have rtx3080 * 8
but it can only running on 1 gpu.
how can i running it on 8 gpus?

Support for official 512 weights

When i try to use the official 512 simswap weights 550000_net_G.pth i get this error:

But it works fine with the unofficial 512 model included on your google drive link

Error trying 512 simswap. loading state_dict

[root@www simswap-inference-pytorch]# python app.py --config-name=run_image.yaml
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
Error executing job with overrides: []
Traceback (most recent call last):
File "/root/simswap-inference-pytorch/app.py", line 68, in main
app = Application(config)
File "/root/simswap-inference-pytorch/app.py", line 49, in init
self.model = SimSwap(
File "/root/simswap-inference-pytorch/src/simswap.py", line 81, in init
self.simswap_net = get_model(
File "/root/simswap-inference-pytorch/src/model_loader.py", line 75, in get_model
model.load_state_dict(state_dict)
File "/usr/local/lib64/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Generator_Adain_Upsample:
Unexpected key(s) in state_dict: "down4.0.weight", "down4.0.bias", "down4.1.weight", "down4.1.bias", "down4.1.running_mean", "down4.1.running_var", "down4.1.num_batches_tracked", "up4.1.weight", "up4.1.bias", "up4.2.weight", "up4.2.bias", "up4.2.running_mean", "up4.2.running_var", "up4.2.num_batches_tracked".

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.

run_image.yaml

data:
id_image: "/root/1.jpg"
att_image: "/root/2.jpg"
specific_id_image: "none"
att_video: "none"
output_dir: "/root"

pipeline:
face_detector_weights: "${hydra:runtime.cwd}/weights/face_detector_scrfd_10g_bnkps.onnx"
face_id_weights: "${hydra:runtime.cwd}/weights/arcface_net.jit"
parsing_model_weights: "${hydra:runtime.cwd}/weights/79999_iter.pth"
simswap_weights: "${hydra:runtime.cwd}/weights/simswap_512_390000_net_G.pth"
gfpgan_weights: "${hydra:runtime.cwd}/weights/GFPGANv1.4_ema.pth"
blend_module_weights: "${hydra:runtime.cwd}/weights/blend_module.jit"
device: "cpu"
crop_size: 512

checkpoint_type: "none" #"official_224" #
face_alignment_type: "none"
smooth_mask_iter: 7
smooth_mask_kernel_size: 17
smooth_mask_threshold: 0.9
face_detector_threshold: 0.6
specific_latent_match_threshold: 0.05
enhance_output: True

defaults:

self
override hydra/hydra_logging: disabled
override hydra/job_logging: disabled

hydra:
output_subdir: null
run:
dir: .

Anyway to remove 'Deepfake' watermark?

I realized that after updating to latest version it puts a watermark at right bottom corner. Anyway to remove it?

does it work with the latest simswap training model ?

I am goint to train a model on 512 , i am seeing that the simswap officall test is complicted , so does this work with the latest model ?
@mike9251

mike9251 / simswap-inference-pytorch Goto Github PK

simswap-inference-pytorch's Introduction

Unofficial Pytorch implementation (inference only) of the SimSwap: An Efficient Framework For High Fidelity Face Swapping

Updates

Attention

Preparation

Installation

Important

Weights

Weights for all models get downloaded automatically.

Inference

Web App

Command line App

Overriding parameters with CMD

Video

License

Acknowledgements

simswap-inference-pytorch's People

Contributors

Stargazers

Watchers

Forkers

simswap-inference-pytorch's Issues

run_image.yaml

Recommend Projects

Recommend Topics

Recommend Org