vsr-duf's People
Forkers
liuguoyou wenlongzhang0517 kanbo0409 wujinlonglovezhangmiao1314 sola303 caotong0 flt19940317 xinntao joeupwu adi-repo abrahamyabo zehaoy liuchongwei marshall-yao astommars jiaweiguan jaso203 gdfzero hqleeustc wallace-he gavin666github lydaidai megayeye ali-mozafari milletzcz jarygrace zhwzhong zhangjf23 undercontroller bencoster zymale pan-22 turgunyusuf dude-with-the-long-username xuleyoung evangelion09 pbdahzou wh-forker easoncyx yuanyuansun-ua arnabmaity dream-winter skerit yangtong1989 aromaticj claforte kirillova-anastasia vidit631 hyunobae jazzylee arslanmclvsr-duf's Issues
Training dataset release
Hi,
May I know if you can release your training data set?
Results in MSU Video Super Resolution Benchmark
Hello,
MSU Video Group has recently launched Video Super Resolution Benchmark and evaluated this algorithm.
DUF-28L takes 5th place by subjective score, 11th place by PSNR, and 9th by our metric ERQAv1.0. DUF-16L takes 8th place by subjective score, 12th place by PSNR, and 11th by our metric ERQAv1.0. You can see the results here.
If you have any other VSR method you want to see in our benchmark, we kindly invite you to participate.
You can submit it for the benchmark, following the submission steps.
Training code && data && Supplementary material
Thanks for sharing your testing code. Do you plan to release your training data and code? And could you share your supplementary material of your paper?
I want to know if the training data needs GT
This algorithm creates artifacts
Hi @yhjo09 I have been testing your algorithm a lot and I notice artifacts in the result images that are not present in the input images.
Your own repo has a good example of what I am talking about. In the result image here you can notice green and black artifacts at the top of the image. This is not present in the original input here
I notice the same thing on my images while trying every single supported combination of <R><L>
values.
On some (i.e. 4 52
) the artifacts are on the left of the image.
Your paper does show full images - did they have the same issue?
If not how did you solve this issue?
Testing with 16L x2 and GT inputs didn't work properly
Hi, just read your great work and thanks for sharing the code.
However, when I tried to run the test code on your data with the arguments python test.py 2 16 G
, the results didn't look well as I show below.
I've tried all the other parameters you offer, and they seem to be good. So I wonder if there's something wrong with 'params_16L_x2.h5'
.
Training code release
Hi,
Just gave your method a try, the core idea is quite elegant and the results look really nice!
I was wondering, are you planning on releasing the training code to allow retraining other networks?
testing data inputs under VSR-DUF/inputs/L/inputs
VSR-DUF/inputs/L/inputs only contain calendar inputs
could you upload other inputs, or offer a link for that?
What are the gaussian filter parameters for x2, x3 ?
Hi~Thanks for your greate work and sharing the codes.
I'm using your project and my own data to train video super-resolution X2、X3 and X4 model,
the X4 model works well, but the X2、X3 models don't .
when running the X2、X3 models, I find that the output residual looks abnormally.
I notice that you comment "h = gkern(13, 1.6) # 13 and 1.6 for x4" for the gaussian filter, but I don't change this parameter in the X2、X3 models.
I am wondering if the improper parameter causes the wrong residual.
Would you please tell me what the gaussian filter parameters that you used for training the X2、X3 models?
Thank you very much!
Some question about making training datasets?
I have read your inspiring paper. How do you select areas of motion in paper's part 4.Implementation? Because I try your method training with randomly cropping the blocks in videos. And the performance is not really good like yours.
residual is always zero
Why 512*288 image frames can't work?
This is the log:
Caused by op 'G/MatMul', defined at:
File "test.py", line 81, in
GH = G(L, is_train)
File "test.py", line 49, in G
t = DynFilter3D(x[:,T_in//2:T_in//2+1,:,:,c], Fx[:,0,:,:,:,:], [1,5,5]) # [B,H,W,RR]
File "/home/disk1/hym/pycharm_remote_files/VSR-DUF/utils.py", line 214, in DynFilter3D
x = tf.matmul(x_localexpand, F) # b, h, w, 1, RR
File "/home/disk1/hym/anaconda/envs/python37/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 2015, in matmu l
a, b, adj_x=adjoint_a, adj_y=adjoint_b, name=name)
File "/home/disk1/hym/anaconda/envs/python37/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 1245, in b atch_mat_mul
"BatchMatMul", x=x, y=y, adj_x=adj_x, adj_y=adj_y, name=name)
File "/home/disk1/hym/anaconda/envs/python37/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 78 7, in _apply_op_helper
op_def=op_def)
File "/home/disk1/hym/anaconda/envs/python37/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in ne w_func
return func(*args, **kwargs)
File "/home/disk1/hym/anaconda/envs/python37/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3272, in crea te_op
op_def=op_def)
File "/home/disk1/hym/anaconda/envs/python37/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1768, in in it
self._traceback = tf_stack.extract_stack()
InternalError (see above for traceback): Blas xGEMMBatched launch failed : a.shape=[147456,1,25], b.shape=[147456,25,16], m=1, n=1 6, k=25, batch_size=147456
[[{{node G/MatMul}} = BatchMatMul[T=DT_FLOAT, adj_x=false, adj_y=false, _device="/job:localhost/replica:0/task:0/device:G PU:0"](G/ExpandDims, G/strided_slice_8)]]
[[{{node G/add_18/_317}} = _Recvclient_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", sen d_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_2365_G/add_18", tensor_type= DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
Code implementation
Hi, I have read the source code and paper.
I have questions about Filter generation network and Residual generation network. Figure 3 in the paper shows that 2d convolution blocks (1x1 and 3x3) follow the denseLayer, however, there are 3d convolution blocks in the code.
Which one should I follow?
Thanks!
Release: practical, user and system friendly version of test.py
- no excessive system memory hogging
- adjustable video memory fraction to maximize video memory utilization (read the file)
- can load all images from the inputdir "at once", no splitting to separate directories to avoid memory hogging
- eyes-convenient representation powered by tqdm
- can take other image formats (such as bmp) directly from the console
- frame-count can be specified directly on console
tested on python 3.6/TF1.15.4
requirements: place the new test_opt.py into the directory of VSR-DUF
pip3 install tqdm
start inferencing:
python test_opt.py 2 16 dataset 0 33492 png
where:
2 = upscale factor
16 = depth
dataset = "./inputs/dataset"
0 = start frame
33492 = end frame
png = image format (any that will work with LoadImage()
limitation: designed for 6-digit frame numbers for file names, to adjust, modify str("%06d"%counter) and (outdir + '/{:06d}.png'.format(counter)) from default 6 to what your movie requires
TODO and near future plans:
improve tqdm representation to work in terms of fps not it/s - done
allow to specify digit count on the console
updated:
- added functionality to operate on a specific range of frames
- added error protection, now testing for existence of first/last frames
- re-designed the progress bar
- variable names friendly to programmer
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.