Your work is awesome! I have tested with your pre-trained model on CelebaTest and got amazing visual results.

However, I noticed that your paper and the VQFR's paper both provide metrics for testing VQFR on CelebaTest, and these metrics are different.

Thus, I used your model to infer on the CelebaTest dataset provided by VQFR link here and use the calculation script provided by VQFR link here, the unexpected results were obtained.

Therefore, I am very curious about the quantitative metrics mentioned in the paper. How do you calculate the metrics? Could you release your metric calculation script please?

I met this problem when I tried to run the command CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --standalone --nproc_per_node=4 --nnodes=1 main_diffusion.py --gpu_id 0123 --cfg_path configs/training/diffusion_ffhq512.yaml --save_dir myfolder. Could someone help me solve it?

Hi, you have done a nice work! I am interested in your paper. Could you please release your pretrain diffusion ckpts on FFHQ?

Hi, I am interested in your nice work, but after reading your paper, I donot understand the way how estimator work.
Therefore, I read your code, and I suppose that the estimator(swinIR) is used to predict the x0 by low quality image y0, and add noise like diffusion forward process do?

im_hq = self.model_ir(y0)

and

yt = self.diffusion.q_sample(
             x_start=post_fun(im_hq),
             t=torch.tensor([start_timesteps,]*im_hq.shape[0], device=device),
)

finally sample by ddpm

sample = self.diffusion.p_sample_loop(
                    self.model,
                    shape=yt.shape,
                    noise=yt,
                    start_timesteps=start_timesteps,
                    clip_denoised=True,
                    denoised_fn=None,
                    model_kwargs=None,
                    device=None,
                    progress=False,
 )

The script I run is as followings. Did I miss any details? ^_^

python inference_difface.py --aligned --in_path testdata/cropped_faces --out_path result/testdata --gpu_id 2

adjust_lr() missing 1 required positional argument: 'ii'adjust_lr() missing 1 required positional argument: 'ii'.

In trainer_py trainer.py", line 255, in train self.adjust_lr()self.adjust_lr()

Please correct me if I am wrong? What to put there? I am trying to train model with batch size [8, 2]

python.exe D:\develop\DifFace\inference_difface.py --in_path D:\develop\DifFace\testdata\whole_imgs --out_path D:\data --gpu_id 0
Setting random seed 20000
Loading from ./weights/diffusion/iddpm_ffhq512_ema500000.pth...
Loaded Done
C:\Users***\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\builder\windows\pytorch\aten\src\ATen\native\TensorShape.cpp:3191.)
return _VF.meshgrid(tensors, kwargs) # type: ignore[attr-defined]
Loading from ./weights/SwinIR/General_Face_ffhq512.pth...
Loaded Done
C:\Users*****\AppData\Local\Programs\Python\Python39\lib\site-packages\torchvision\models_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
warnings.warn(
C:\Users*\AppData\Local\Programs\Python\Python39\lib\site-packages\torchvision\models_utils.py:223: UserWarning: Arguments other than a weight enum or None for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing weights=None.
warnings.warn(msg)
Traceback (most recent call last):
File "D:\develop\DifFace\inference_difface.py", line 160, in
main()
File "D:\develop\DifFace\inference_difface.py", line 137, in main
image_restored, face_restored, face_cropped = sampler_dist.sample_func_bfr_unaligned(
File "D:\develop\DifFace\sampler.py", line 368, in sample_func_bfr_unaligned
restored_faces = _process_batch(self.face_helper.cropped_faces)
File "D:\develop\DifFace\sampler.py", line 332, in _process_batch
restored_faces = self.sample_func_ir_aligned(
File "D:\develop\DifFace\sampler.py", line 279, in sample_func_ir_aligned
sample = self.diffusion.p_sample_loop(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 428, in p_sample_loop
for sample in self.p_sample_loop_progressive(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 484, in p_sample_loop_progressive
out = self.p_sample(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 383, in p_sample
out = self.p_mean_variance(
File "D:\develop\DifFace\models\respace.py", line 88, in p_mean_variance
return super().p_mean_variance(self._wrap_model(model), *args, **kwargs)
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 278, in p_mean_variance
min_log = _extract_into_tensor(
File "D:\develop\DifFace\models\gaussian_diffusion.py", line 105, in _extract_into_tensor
res = th.from_numpy(arr).to(device=timesteps.device)[timesteps].float()
KeyboardInterrupt

Thank you for your great Job!!

You updated this repo in 2023/12/12 maybe, so there is no links for SwinIR model
https://github.com/zsyOAOA/DifFace/releases/download/V1.0/General_Face_ffhq512.pth

and now, I downloaded it(below link) from your release page instead of before link
https://github.com/zsyOAOA/DifFace/releases/download/V1.0/swinir_restoration512_L1.pth
but, I cant load the model for running inference.py

This file is not available for model training

I trained SwinIR and saved the ckpts .
i'm trying to inference the new model after i changed the ckpts paths in "iddpm_ffhq512_swinir.yaml"
i have issue in loading new model state_dict

" RuntimeError: Error(s) in loading state_dict for SwinIR:
Missing key(s) in state_dict: "conv_first.1.weight", "conv_first .... "

note : " inference code is worked with pretrained "model /weights/SwinIR/General_Face_ffhq512.pth" .

How long it takes to train the entire network?

Thank for sharing your great work!! : )

We have tried to training DifFace with SwinIR diffused estimator and got an error that self.loss_fun(line 492, 495 in trainer.py) is not defined.

        if last_batch or self.num_gpus <= 1:
            loss = self.loss_fun(hq_pred, micro_data['gt']) / hq_pred.shape[0]
        else:
            with self.model.no_sync():
                loss = self.loss_fun(hq_pred, micro_data['gt']) / hq_pred.shape[0]

=======================
Please check these lines.

I met this problem when I tried to run the command CUDA_VISIBLE_DEVICES=0 python main_sr.py --cfg_path configs/training/swinir_ffhq512.yaml --save_dir save_dir_1 Could someone help me solve it?

Have you ever tried to decrease the lr(the default value is 1e-4) for the training procedure of diffusion model?
Since the loss converges very quick(less than 1W iters), but the lr keeps constant during the whole training time.

@zsyOAOA how to train FFHQ on 1024x1024 resolution. I have checked the swinIR_ffhq512 the img_size : 64 (at line8).

Could you add a device number to the command line parameters?

followed the example code from the huggingface model card but got the error

then tried the below with the same error

from diffusers import DiffusionPipeline
model_id = "OAOA/DifFace"

# load model and scheduler
difface = DiffusionPipeline.from_pretrained(model_id)  

i'm using diffusers==0.25.0

Number of parameters: 15.79M
=> Loaded checkpoint /home/zhuchao/code/diface/code/diface/weights/SwinIR/General_Face_ffhq512.pth
Traceback (most recent call last):
File "main_sr.py", line 32, in
trainer = Trainer(configs)
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 285, in init
super().init(configs)
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 67, in init
self.resume_from_ckpt()
File "/home/zhuchao/code/diface/code/diface/trainer.py", line 145, in resume_from_ckpt
util_net.reload_model(self.model, ckpt['state_dict'])
KeyError: 'state_dict'
I want to finetune swinIR on a small face datasets, but I met this error.
It seems the General_Face_ffhq512.pth not save state_dict and iters_start，I don't know how to fix it.
Hope you can help me, THANKS!

I retrained the Train diffused estimator (SwinIR) stage and found that the saved weights are dict_keys(['iter_start','log_step','log_step_img','state_dict']) and not SwinIR's weights, so the inference reports a load model error. Would like to ask if the code in main_sr.py about saving model weights is wrong? I'm looking forward to hearing from the author!

By running this command python inference_difface.py --in_path ~/Images/blur/blurred_1.png --out_path ~/Images/blur/unblurred I got error

Here the python command line stack Trace

Traceback (most recent call last):
  File "inference_difface.py", line 160, in <module>
    main()
  File "inference_difface.py", line 137, in main
    image_restored, face_restored, face_cropped = sampler_dist.sample_func_bfr_unaligned(
  File "~/programmation/DifFace/sampler.py", line 368, in sample_func_bfr_unaligned
    restored_faces = _process_batch(self.face_helper.cropped_faces)
  File "~/programmation/DifFace/sampler.py", line 328, in _process_batch
    cropped_face_t = np.stack(
  File "<__array_function__ internals>", line 180, in stack
  File "~/anaconda3/envs/DifFace/lib/python3.8/site-packages/numpy/core/shape_base.py", line 422, in stack
    raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

I have CUDA installed, here the output of nvidia-smi command

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.105.17   Driver Version: 525.105.17   CUDA Version: 12.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| N/A   42C    P8    N/A /  N/A |      4MiB /  4096MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A       952      G   /usr/lib/xorg/Xorg                  4MiB |
+-----------------------------------------------------------------------------+

Hi, thanks for sharing your code! Could you please tell me what and how many GPUs you used and how long you trained for?Thanks!