the-learning-and-vision-atelier-lava / sof-vsr Goto Github PK

Hi,

I'm trying to run SOF-VSR on a machine with 24gb VRAM and 64gb CPU RAM.

In CPU-only mode, I'm running into issues running the model on input images with resolutions larger than HD (1920 x 1080). In GPU-only mode, I can't run the model on input images with resolutions greater than 960 pixels.

Is this in-line with your research and findings? What input resolutions are possible with SOF-VSR? Or, is it only limited by the amount of hardware on your system? Thanks so much!

hi @LongguangWang
thank you for your work
could you please show me how to run your code with cpu only, i try to add --cpu arg to demo_Vid4.py as parser.add_argument("--cpu", action="store_true",help="use cpu only") , then i run the test with
python demo_Vid4.py --video_name calendar --upscale_factor 4 --cpu
but it give me this error
Traceback (most recent call last):
File "demo_Vid4.py", line 54, in
main(cfg)
File "demo_Vid4.py", line 30, in main
net.cuda()
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 260, in cuda
return self._apply(lambda t: t.cuda(device))
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 187, in _apply
module._apply(fn)
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 187, in _apply
module._apply(fn)
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 193, in _apply
param.data = fn(param.data)
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\nn\modules\module.py", line 260, in
return self.apply(lambda t: t.cuda(device))
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\cuda_init.py", line 161, in _lazy_init
check_driver()
File "C:\Users\ABDO\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\cuda_init.py", line 75, in _check_driver
raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
thank you

Thank for your excellent work. It helps me a lot. I want to evaluate the consistency performance of my results . How do you implement these evaluation metrics:T-MOVIE, MOVIE and VQM-VFD? Is there source code package to use?

The result of 31 frames of test video is only 29 frames, and the frames at the beginning and end of the video can't be processed?

Hello,

MSU Graphics & Media Lab Video Group has recently launched two new Super-Resolution Benchmarks.

• Video Upscalers Benchmark: Quality Enhancement determines the best upscaling methods for increasing video resolution and improving visual quality.
• Super-Resolution for Video Compression benchmark aims to test Super-Resolution methods on compressed videos and select the best model for each video codec standard.

Your method achieved 8th place in Super-Resolution for Video Compression Benchmark in 'x264 compression' category. We congratulate you on your result and look forward to your future work!

We would be grateful for your feedback on our work.

Hi, @LongguangWang
Thanks your excellent work!
Will you release code and dataset for training ?

Hi, thanks for your great work,
where are the pretrained models you mentioned in readme

Hi, thanks for you wonderful work and open source.

After i retrained the code , i got the bad test result on vid4 dataset. Its psnr is 18.64,while the counterpart on paper is 26.01.

I think may be the two reasons:

1. training set
   Since the CVDL dataset is to be processed to get training set . So my approach may be different from yours, which leads to this problem. Can you provide a well-processed training set or code of getting training set from CVDL dataset?

Is the published code different from what you trained before?

Best regard,

Hello,
MSU Video Group has recently launched Video Super Resolution Benchmark and evaluated this algorithm.

SOF-VSR-BD takes 11th place by subjective score, 10th place by PSNR, and 8th by our metric ERQAv1.0. SOF-VSR-BI takes 12th place by subjective score, 5th place by PSNR, and 7th by our metric ERQAv1.0. You can see the results here.

If you have any other VSR method you want to see in our benchmark, we kindly invite you to participate.
You can submit it for the benchmark, following the submission steps.

According to the paper, the draft cube should be HxWx(2ss+1) instead of HxWx(2ss+3) in the code, which concatenates not just the central LR frame and can be fixed by changing the following line
https://github.com/LongguangWang/SOF-VSR/blob/420390814ec5d0c3af6678b0773e87ca13279853/modules.py#L172
to
"draft_cube = torch.unsqueeze(x[:, 1, :, :], dim=1)".
Does it matter to the performance?

Hello, thanks for your nice work on temporal consistency!

I tried using demo_vid4.py to generate SR videos on vid4 and get a worse result(in terms of temporal consistency, there are lots of flickering artifacts) compared which reported in paper, as follows:

temporal profile:

calendar gif:
calendar

Could you give me some hints? Thanks in advance!

非常感谢您的工作。

测试是不知道为什么会丢失掉第一个SR图和最后一个SR图。

非常感谢！

Hi, Nice work

I found a confusing part of your TestsetLoader.

You normalize the LR tensors twice. One is in the code /255, then .div(255) again in toTensor()

I don't think it make sense.

Hi,

I noticed that when running SOF-VSR on an input frame sequence with N frames, the output has (N - 2) frames. It doesn't produce the first and last frames in the sequence. What is the best way to get around this limitation? Should we duplicate the first and last frames of the input sequence before running the model? Thanks!

Hello,

Nice work and code!

I have a question regarding the optical_flow_warp method. I understand that we want grid to have values in the range of [-1,1], but the multiplication with 31 seems arbitrary to me.

    flow_0 = torch.unsqueeze(image_optical_flow[:, 0, :, :] * 31 / (w - 1), dim=1)
    flow_1 = torch.unsqueeze(image_optical_flow[:, 1, :, :] * 31 / (h - 1), dim=1)

Can somebody further explain where this comes from?

Hi,

Is the data that was used for training hosted anywhere?

I understand that the data was taken from https://www.cdvl.org/, but which videos were used?

Hi @LongguangWang
i was wonder if it possible to add chop_forward to this great project to avoid run out of memory and Freezing the pc,i don't know if it even possible with this kind of Video Super-Resolution.
thank you

首先感谢你的工作
今天下载以后，使用Pytorch 0.4.1，在cpu模式下运行

报错内容：
Traceback (most recent call last):
File "demo_Vid4.py", line 53, in
main(cfg)
File "demo_Vid4.py", line 48, in main
os.mkdir('results/' + video_name)
FileNotFoundError: [WinError 3] 系统找不到指定的路径。: 'results/calendar'

使用管理员模式运行也不行，请问应该怎么处理呢