Can you help me to reproduce your result? about fastdvdnet HOT 31 OPEN

I also have the similar problem and my results seem to quite similar to some of the results mentioned here i.e I got 31.257 PSNR on noise=30 on set8.

from fastdvdnet.

even when i run the trained model the results are not completely similar. I get 31.60 something vs 31.68 mentioned in the paper.

from fastdvdnet.

Hey sorry i forgot to inform you . The above command works for me. Thank you so much

from fastdvdnet.

I fixed some problem in my "set8".
Now I get 29.46dB in "set8" with your weights. (paper is 29.53dB/ +0.07dB )
I think this difference is reasonable because seed for noise is not fixed.

But still my weight shows 29.09dB in fixed set8 (Your's:29.46 dB / +0.37dB ).
It is non-trivial difference.

Is there any tip to get your result?

from fastdvdnet.

Hi dustlrdk, sorry for the late response, I've been a bit busy.
I'm uploading the whole Set8 testset to the folder linked in the README file so you can try running the algorithm with them.

By the way, the PSNR displayed on the paper is the average of the PNSR of each sequence in the testset. The PSNR of each sequence is the average of the PSNR of all frames.

So for your second question, as far as I understand, you try to train the model and you get different results. As you know, for the training you need mp4 files with the davis sequences. When converting the sequences one has to pay particular attention to the 'crf' and 'keyint' ffmpeg parameters to avoid strong compression. For the code to convert the image sequences see this gist
https://gist.github.com/m-tassano/0536391eb79d63864e5005ea4da88243

hope this helps

from fastdvdnet.

Thank you for your sincere reply.

I re-run test code with uploaded data.
I got a result(our 29.4586dB | your paper 29.53dB).(You said you used "macro averages," didn't you?)

dir=./data/$1

for f in "$dir"/;
do
i=${f##/}
CUDA_VISIBLE_DEVICES=$3 python test_fastdvdnet.py --test_path $f --noise_sigma $2 --save_path results/$1/sigma$2/$i/ --max_num_fr_per_seq 85
done

for f in "$dir"/;
do
i=${f##/}
cat results/$1/sigma$2/$i/log.txt | grep PSNR | cut -f9-9 -d" " | cut -f1 -d"d"
done

Can you try this shell code with your code and weight?

I got this.

30.2415
26.7901
26.9022
28.791
28.3301
33.0974
31.9209
29.5933

Average : 29.4583125

And I used mp4 files in training folder you uploaded on the dropbox, do I have to make new mp4 training files instead?

from fastdvdnet.

I use the training mp4 from your dropbox, and set all the parameters the same as your paper mentioned using your training code. I have trained at least ten models but none of them achieve the performance of your provided model or the results on your paper.
The set8 results with sigma 50 are as follows (the format is * test sequence name: PSNR of the provided model.pth vs PSNR of my trained model):
* hypersmooth: 30.2355 vs 30.1125
* motorbike: 26.7900 vs 26.7176
* park_joy: 26.9008 vs 26.3757
* rafting: 28.7940 vs 28.6722
* snowboard: 28.3337 vs 28.1645
* sunflower: 33.0950 vs 32.2683
* touchdown: 31.9184 vs 31.3565
* tractor: 29.5916 vs 28.9817
My results on DAVIS-2017-test-dev-480p are also unsatisfying. I list here for comparison (the format is sigme=value: PSNR in your paper vs PSNR of the provided model.pth vs PSNR of my trained model):
sigma=10: 38.71 vs 39.2444 vs 36.1642
sigma=20: 35.77 vs 36.1032 vs 35.7380
sigma=30: 34.04 vs 34.2924 vs 33.8718
sigma=40: 32.82 vs 33.0229 vs 36.5660
sigma=50: 31.86 vs 32.0329 vs 31.5659
And I find the model is tend to get abnormally low PSNR with sigma 10, some of my models even get results like 29.5427 and 33.4307.
Anyone has reproduced the results?
I really can't figure out what's wrong with my training process.

from fastdvdnet.

And using my own training mp4 gets higher results than the mp4 downloaded from your dropbox.
I produce the mp4 using the code "ffmpeg -f image2 -i %05d.jpg xxx.mp4".

from fastdvdnet.

@djmth Anyone has reproduced the results?

Our results seem quite similar. I too have abnormally low performance on sigma=10 (even worse than yours).

@m-tassano

1. The training dataset provided in your DropBox is actually incomplete (only 86 sequences of 90 are present). I proceeded to recreate the training dataset by downloading from DAVIS's official site and then using your gist with the default parameters to create it.
2. My Set8 is the same as your dropbox link (4 folders from gopro and 4 folders from derf).

I am unable to reproduce your results, and this feels to be a prevalent issue. I request you, if possible, as the author, try to confirm these by recreating your experiments from this repo and maybe there is indeed a possible bug somewhere in this repo.

from fastdvdnet.

Hi,

I agree that those PSNRs look abnormal. The reason of this performance might come from a training which did not converge well. Are those the results of only one training or do you see those figures repeating along different experiments on your side?

Those 4 sequences were removed because their resolution is different from the rest of the sequences (shooting:1152x480, disc-jockey:1138x480, cat-girl:911x480, bike-packing:910x480). The reason behind this is that DALI does not support inputs with different resolutions, or at least the DALI version used when I first uploaded the project. That means, the provided training dataset is correct and the one used to train the provided weights.

Another factor which might introduce variations is the way you compute the PSNRs of each frame and how you accumulate the scores of all frames of all sequences. You can find the script I used to compute the metrics and the tables in the paper [here|https://www.dropbox.com/s/wei8uhym1i4viot/make_tables_allalgos.ipynb?dl=0]. You are welcome to use the code with your results.

Lastly, I was recently forced to update the supported DALI version because the original version which was employed when the method was published is no longer supported by NVIDIA. I have not tested this new DALI version thoroughly and it might have an impact on the results. I will try to run a training in the following days if I can find the time.

from fastdvdnet.

Hello. Thanks for your reply!

Are those the results of only one training

My results are from a single experiment.
Although, since most results reported by others on this repo seem similar, I feel we could rule out the possibility of a single bad experiment.

the provided training dataset is correct and the one used to train the provided weights

Thanks for this clarification! Oh by the way, the dataloader from the new DALI version was able to load these anomalous sequences as well!

You can find the script I used to compute the metrics and the tables in the paper

Great. I shall try computing metrics using this script.

I will try to run a training in the following days if I can find the time

Thank you very much. I really appreciate it.

from fastdvdnet.

@m-tassano How were the run-times calculated?. Was it similar to the macro-averages in the case of PSNRs (average of averages of a video sequence)?

If it was in some other way is it possible to share the script/notebook/method you used to calculate the run-times.

Edit: Also, which test sequence did you use to calculate run-times? Set-8 or DAVIS-Test?

from fastdvdnet.

Hello @m-tassano
I am trying to train model with same training mp4 files but after 1epoch it gives me ZeroDivisionError. I kept validation images in validation folder but still gives this error. can you help me to solve this problem. I am using your updated release code.

File "train_fastdvdnet.py", line 212, in
main(**vars(argspar))
File "train_fastdvdnet.py", line 147, in main
trainimg=img_train
File "/home/knuvi/Desktop/Kavita/fastdvdnet-0.1/train_common.py", line 130, in validate_and_log
psnr_val /= len(dataset_val)
ZeroDivisionError: division by zero

from fastdvdnet.

Hi @kavita19
From reading the error you sent, it seems your problem comes from the fact that the length of the validation dataset is zero. That might might happen for example if you're passing an incorrect path to the --valset_dir in train_fastdvdnet.py so that when dataset_val in line 31 (dataset_val = ValDataset(valsetdir=args['valset_dir'], gray_mode=False)) no images are found. The argument dataset_val needs to be set to the path of the folder with the validation images.

from fastdvdnet.

@m-tassano @Syzygianinfern0
For validation images i have added 15 images (own images with size 960x540) in my validation dir but when I run train_fastdvdnet.py I am getting below problem now. can you help me to solve this. Also i bit confused about testing images (how many images per sequence tests this model)?

/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/base_iterator.py:162: Warning: Please set reader_name and don't set last_batch_padded and size manually whenever possible. This may lead, in some situations, to missing some samples or returning duplicated ones. Check the Sharding section of the documentation for more details.
_iterator_deprecation_warning()
Traceback (most recent call last):
File "train_fastdvdnet.py", line 212, in
main(**vars(argspar))
File "train_fastdvdnet.py", line 38, in main
temp_stride=3)
File "/home/knuvi/Desktop/Kavita/fastdvdnet-0.1/dataloaders.py", line 110, in init
auto_reset=True)
File "/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/pytorch.py", line 183, in init
self._first_batch = DALIGenericIterator.next(self)
File "/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/pytorch.py", line 194, in next
outputs = self._get_outputs()
File "/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/base_iterator.py", line 255, in _get_outputs
outputs.append(p.share_outputs())
File "/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/pipeline.py", line 863, in share_outputs
return self._pipe.ShareOutputs()
RuntimeError: Critical error in pipeline:
Unknown error
Current pipeline object is no longer valid.

from fastdvdnet.

Current pipeline object is no longer valid.

I have had this error when I did not have the CUDA environment properly set up, but I am not sure if that is the only cause of the issue. Make sure your installed CUDA version matches the DALI CUDA version.

from fastdvdnet.

https://docs.nvidia.com/deeplearning/dali/user-guide/docs/installation.html#nvidia-dali

You can choose the DALI CUDA version you want to install here 👆

from fastdvdnet.

@m-tassano @Syzygianinfern0
ok my current version of cuda is 11.4 and I install cuda for 11.
no I am getting cuda out of memory problem. Do u have any solution this. because I tried by reducing batch size but it doesn't work.

/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/base_iterator.py:161: Warning: Please set reader_name and don't set last_batch_padded and size manually whenever possible. This may lead, in some situations, to missing some samples or returning duplicated ones. Check the Sharding section of the documentation for more details.
_iterator_deprecation_warning()
Traceback (most recent call last):
File "train_fastdvdnet.py", line 212, in
main(**vars(argspar))
File "train_fastdvdnet.py", line 38, in main
temp_stride=3)
File "/home/knuvi/Desktop/Kavita/fastdvdnet-0.1/dataloaders.py", line 110, in init
auto_reset=True)
File "/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/pytorch.py", line 190, in init
self._first_batch = DALIGenericIterator.next(self)
File "/home/knuvi/anaconda3/envs/fastdvdnet2/lib/python3.6/site-packages/nvidia/dali/plugin/pytorch.py", line 238, in next
device=category_device[category])
RuntimeError: CUDA error: out of memory

from fastdvdnet.

What is your GPU memory capacity? Is it out of memory even with batch size of 1?

I remember being able to have the code run even on my laptop GPU which is just 4 GB.

from fastdvdnet.

@Syzygianinfern0 Thanks. My cuda memory problem is solved after restarting.

How many validation images we can test ? because I have converted .yuv video format to .png images (total 100 frames for validation). After setting my environment I started training but after one epoch I am getting zerobydivision error.

[epoch 1][1981/2000] loss: 15.6695 PSNR_train: 0.0000
[epoch 1][1991/2000] loss: 15.7777 PSNR_train: 0.0000
Traceback (most recent call last):
File "train_fastdvdnet.py", line 212, in
main(**vars(argspar))
File "train_fastdvdnet.py", line 147, in main
trainimg=img_train
File "/home/knuvi/Desktop/Kavita/fastdvdnet-0.1/train_common.py", line 130, in validate_and_log
psnr_val /= len(dataset_val)
ZeroDivisionError: division by zero

this is command i am giving to for training model but I dont know why My validation images not getting exact path thats why this error occur.

(fastdvdnet2) knuvi@DGX-Station:~/Desktop/Kavita/fastdvdnet-0.1$ python train_fastdvdnet.py --batch_size 128 --epochs 80 --patch_size 96 --trainset_dir "/home/knuvi/Desktop/Kavita/fastdvdnet-0.1/img/train/" --valset_dir "/home/knuvi/Desktop/Kavita/fastdvdnet-0.1/img/basketball100/" --log_dir logs

from fastdvdnet.

@kavita19

.yuv video format to .png images

Is this some custom dataset?

You can first try just trying to run the code using the provided training and validation datasets. And if that works, you can compare how your custom dataset's structure is looking like with respect to the provided validation dataset structure.

By the way, the validation set directory should point to a folder containing multiple folders which contain images with file names in sequential order.

from fastdvdnet.

@Syzygianinfern0 Ok for the validation set which images you have used? Because I Have tried with Derf 480p testset ("tractor") but it gives me same problem. Also the image should be jpg or png ? and I save my images in sequential order but when I put in validation folder it shuffles.

from fastdvdnet.

Are you using the command this way --valset_dir img/?

Please refer to the last sentence here: #13 (comment)

from fastdvdnet.

I am using --valset_dir "./home/knuvi/Desktop/Kavita/fastdvdnet-0.1/img/tractortest"

Is it wrong ?can u share me your command for training that you have used?

from fastdvdnet.

Can you please try it as --valset_dir "./home/knuvi/Desktop/Kavita/fastdvdnet-0.1/img/"

from fastdvdnet.

@m-tassano In fastdvdnet model (Model.py) number of input are 5 and it takes 3 frames to pass denoising blocks. Similarly if I want to takes only 2frames from inputs is it possible and what changes required in model.py?

from fastdvdnet.

@kavita19 please create a new issue for a separate topic

from fastdvdnet.

@m-tassano @Syzygianinfern0
As I read in the paper runtimes shows 0.1s to denoise image. But when I tested snowboard sequence runtime is (Denoised 59 frames in 3.932s, loaded seq in 6.258s). Please can you clear me paper runtime result is based on one image or all frames? and which dataset you used to calculate runtime.

from fastdvdnet.

@kavita19 I'll try to answer to your question the best I can because I find it's not completely clear.

As mentioned in the paper, the reported time is the time it takes to denoise a color frame of resolution 960×540. So, it doesn't really matter which dataset you use as long as its frames have this resolution--I believed I used one of the sequences in the Set8 dataset, in any case.

The runtime estimation was computed as the total time over the number of frames in the sequence. The same was used for all the algorithms mentioned. Note that if you denoise a sequence with more frames, the runtime per frame will likely decrease, as there's an overhead when loading the input sequence.

Hope this helps

from fastdvdnet.

@m-tassano Thank you for your reply.
for validation which dataset you used and how many png images you were used in validation ?

from fastdvdnet.

Please note that the performance of the model trained with the old DALI version (the one originally used to train the weights shared in the Github) has been reported to be superior than the one obtained with the new DALI version.
Or inversely, the new DALI version is linked to a drop in the performance of the model.
See #51 for more details.

from fastdvdnet.