Code Monkey home page Code Monkey logo

difface's People

Contributors

eltociear avatar jaywu109 avatar madoibito80 avatar zsyoaoa avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

difface's Issues

Does adjustment of lr work?

Have you ever tried to decrease the lr(the default value is 1e-4) for the training procedure of diffusion model?
Since the loss converges very quick(less than 1W iters), but the lr keeps constant during the whole training time.

adjust_lr() missing 1 required argument

adjust_lr() missing 1 required positional argument: 'ii'adjust_lr() missing 1 required positional argument: 'ii'.

In trainer_py trainer.py", line 255, in train self.adjust_lr()self.adjust_lr()

Please correct me if I am wrong? What to put there? I am trying to train model with batch size [8, 2]

About ckpts

Hi, you have done a nice work! I am interested in your paper. Could you please release your pretrain diffusion ckpts on FFHQ?

training time

Hi, thanks for sharing your code! Could you please tell me what and how many GPUs you used and how long you trained for?Thanks!

Model weight preservation in the Train diffused estimator (SwinIR) stage.

I retrained the Train diffused estimator (SwinIR) stage and found that the saved weights are dict_keys(['iter_start','log_step','log_step_img','state_dict']) and not SwinIR's weights, so the inference reports a load model error. Would like to ask if the code in main_sr.py about saving model weights is wrong? I'm looking forward to hearing from the author!

About the guidance function of estimator

Hi, I am interested in your nice work, but after reading your paper, I donot understand the way how estimator work.
Therefore, I read your code, and I suppose that the estimator(swinIR) is used to predict the x0 by low quality image y0, and add noise like diffusion forward process do?

im_hq = self.model_ir(y0)

im_hq = self.model_ir(y0)

and

yt = self.diffusion.q_sample(

yt = self.diffusion.q_sample(
             x_start=post_fun(im_hq),
             t=torch.tensor([start_timesteps,]*im_hq.shape[0], device=device),
)

finally sample by ddpm

sample = self.diffusion.p_sample_loop(

sample = self.diffusion.p_sample_loop(
                    self.model,
                    shape=yt.shape,
                    noise=yt,
                    start_timesteps=start_timesteps,
                    clip_denoised=True,
                    denoised_fn=None,
                    model_kwargs=None,
                    device=None,
                    progress=False,
 )

The script I run is as followings. Did I miss any details? ^_^

python inference_difface.py --aligned --in_path testdata/cropped_faces --out_path result/testdata --gpu_id 2

retrain swinIR based on pretrained model, but met ERROR [KeyError: 'state_dict']

Number of parameters: 15.79M
=> Loaded checkpoint /home/zhuchao/code/diface/code/diface/weights/SwinIR/General_Face_ffhq512.pth
Traceback (most recent call last):
File "main_sr.py", line 32, in
trainer = Trainer(configs)
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 285, in init
super().init(configs)
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 67, in init
self.resume_from_ckpt()
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 145, in resume_from_ckpt
util_net.reload_model(self.model, ckpt['state_dict'])
KeyError: 'state_dict'
I want to finetune swinIR on a small face datasets, but I met this error.
It seems the General_Face_ffhq512.pth not save state_dict and iters_start,I don't know how to fix it.
Hope you can help me, THANKS!

self.loss_fun in trainer.py is not defined.

Thank for sharing your great work!! : )

We have tried to training DifFace with SwinIR diffused estimator and got an error that self.loss_fun(line 492, 495 in trainer.py) is not defined.

        if last_batch or self.num_gpus <= 1:
            loss = self.loss_fun(hq_pred, micro_data['gt']) / hq_pred.shape[0]
        else:
            with self.model.no_sync():
                loss = self.loss_fun(hq_pred, micro_data['gt']) / hq_pred.shape[0]

=======================
Please check these lines.

使用现有模型,测试,返回错误,如何解决?

python.exe D:\develop\DifFace\inference_difface.py --in_path D:\develop\DifFace\testdata\whole_imgs --out_path D:\data --gpu_id 0
Setting random seed 20000
Loading from ./weights/diffusion/iddpm_ffhq512_ema500000.pth...
Loaded Done
C:\Users***\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3191.)
return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined]
Loading from ./weights/SwinIR/General_Face_ffhq512.pth...
Loaded Done
C:\Users*****\AppData\Local\Programs\Python\Python39\lib\site-packages\torchvision\models_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
C:\Users*
\AppData\Local\Programs\Python\Python39\lib\site-packages\torchvision\models_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None.
warnings.warn(msg)
Traceback (most recent call last):
File "D:\develop\DifFace\inference_difface.py", line 160, in
main()
File "D:\develop\DifFace\inference_difface.py", line 137, in main
image_restored, face_restored, face_cropped = sampler_dist.sample_func_bfr_unaligned(
File "D:\develop\DifFace\sampler.py", line 368, in sample_func_bfr_unaligned
restored_faces = _process_batch(self.face_helper.cropped_faces)
File "D:\develop\DifFace\sampler.py", line 332, in _process_batch
restored_faces = self.sample_func_ir_aligned(
File "D:\develop\DifFace\sampler.py", line 279, in sample_func_ir_aligned
sample = self.diffusion.p_sample_loop(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 428, in p_sample_loop
for sample in self.p_sample_loop_progressive(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 484, in p_sample_loop_progressive
out = self.p_sample(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 383, in p_sample
out = self.p_mean_variance(
File "D:\develop\DifFace\models\respace.py", line 88, in p_mean_variance
return super().p_mean_variance(self._wrap_model(model), *args, **kwargs)
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 278, in p_mean_variance
min_log = _extract_into_tensor(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 105, in _extract_into_tensor
res = th.from_numpy(arr).to(device=timesteps.device)[timesteps].float()
KeyboardInterrupt

Could you release your metric calculation script please?

Your work is awesome! I have tested with your pre-trained model on CelebaTest and got amazing visual results.

However, I noticed that your paper and the VQFR's paper both provide metrics for testing VQFR on CelebaTest, and these metrics are different.

Thus, I used your model to infer on the CelebaTest dataset provided by VQFR link here and use the calculation script provided by VQFR link here, the unexpected results were obtained.

Therefore, I am very curious about the quantitative metrics mentioned in the paper. How do you calculate the metrics? Could you release your metric calculation script please?

Can't load diffused estimator (SwinIR) trained model

I trained SwinIR and saved the ckpts .
i'm trying to inference the new model after i changed the ckpts paths in "iddpm_ffhq512_swinir.yaml"
i have issue in loading new model state_dict

" RuntimeError: Error(s) in loading state_dict for SwinIR:
Missing key(s) in state_dict: "conv_first.1.weight", "conv_first .... "

note : " inference code is worked with pretrained "model /weights/SwinIR/General_Face_ffhq512.pth" .

Can't run inference_difface.py

By running this command python inference_difface.py --in_path ~/Images/blur/blurred_1.png --out_path ~/Images/blur/unblurred I got error

Here the python command line stack Trace

Traceback (most recent call last):
  File "inference_difface.py", line 160, in <module>
    main()
  File "inference_difface.py", line 137, in main
    image_restored, face_restored, face_cropped = sampler_dist.sample_func_bfr_unaligned(
  File "~/programmation/DifFace/sampler.py", line 368, in sample_func_bfr_unaligned
    restored_faces = _process_batch(self.face_helper.cropped_faces)
  File "~/programmation/DifFace/sampler.py", line 328, in _process_batch
    cropped_face_t = np.stack(
  File "<__array_function__ internals>", line 180, in stack
  File "~/anaconda3/envs/DifFace/lib/python3.8/site-packages/numpy/core/shape_base.py", line 422, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

I have CUDA installed, here the output of nvidia-smi command

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   42C    P8    N/A /  N/A |      4MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       952      G   /usr/lib/xorg/Xorg                  4MiB |
+-----------------------------------------------------------------------------+

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.