jiaxi-jiang / fbcnn Goto Github PK

请问作者，主页的样例上的退化图是由由Jpeg退化函数从高清图获取的吗？我想获得类似的退化图像具体应该怎么操作？

Hi,

The README says that code will be released here https://github.com/cszn/KAIR/

Any idea when?

Thanks!

Thank you so much for your impressive work, I tried many tools but their performance were not like yours. But I spent a lot of time to figure out how I can use test_split() but I did not figure out where I should use it, could you please tell me in which line I should use it in main_test_fbcnn_color_real.py what should be the input parameters.

          Hello, thanks for your interest in our work! 

1. The function you mentioned is mainly used for generating small patches for fast training. For testing large images, please try to utilize the function test_split from utils_model.py: https://github.com/jiaxi-jiang/FBCNN/blob/main/utils/utils_model.py#L214

2. For fair comparisons, we follow the convention of prior work to use 8 bit depth image for training and testing. Intuitively training with 16 bit depth images should be better for testing 16 bit depth images, but I think not necessary for testing 8 bit depth images. We can discuss more if you are interested in this part.

Feel free to ask me if you still have questions.

Originally posted by @jiaxi-jiang in #12 (comment)

where is the train dataset?

AssertionError: ../trainsets/Data_DIV2K_Flickr2K is not a valid directory在训练过程中出现这样一个问题，请问作者训练数据集的架构分布是如何的。

I'm excited to watch the demo video.
Please release the code for Google colab so that we can try it easily.
I am looking forward to your next update.
Thank you.

Hi, I test the result on classic5 and live1 in gray with pre-trained weight

However, I find some mistakes caused by rgb2ycbcr function

Here is my offline test result

where classic5 is ok because it is gray input, and the live1 result is strange because of the rgb2ycbcr function.

The GT I compare is produced by Matlab version YCbCr, which is different from OpenCV version which is utilized in your training and eval process.
if n_channels == 3: img_L = cv2.cvtColor(img_L, cv2.COLOR_RGB2BGR) _, encimg = cv2.imencode('.jpg', img_L, [int(cv2.IMWRITE_JPEG_QUALITY), quality_factor]) img_L = cv2.imdecode(encimg, 0) if n_channels == 1 else cv2.imdecode(encimg, 3)

The logging result in live1_qf10 is
21-12-23 15:43:59.031 : Average PSNR/SSIM/PSNRB - live1_fbcnn_gray_10 -: 28.96$\vert$0.8254$\vert$28.64.
which is a little higher than 26.97 but much lower than 29.75, because the model is for OpenCV version Y, not Matlab version

However, the common setting in deblocking is using Matlab version Y

I test ICCV21-Learning Dual Priors for JPEG Compression Artifacts Removal, the offline results are

It is similar to the results produced in paper

I hope it may be helpful.

ruti

thanks

Greetings,
Sorry for bothering, but how can I run the trained model?
I tried to create another .py script for trained color.pth model (similar to Color/Real scripts)
But getting an error when tryin to run it.

File "main_test_fbcnn_latest_g.py", line 105, in
main()
File "main_test_fbcnn_latest_g.py", line 55, in main
model.load_state_dict(torch.load(model_path), strict=True)
File "D:\Users\Vasily\anaconda3\envs\FBCNN\lib\site-packages\torch\serialization.py", line 593, in load
return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
File "D:\Users\Vasily\anaconda3\envs\FBCNN\lib\site-packages\torch\serialization.py", line 763, in _legacy_load
magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: could not find MARK

How to solve this issue, could you, please, help?
Thank you in advance~

Sorry to bother you, but could you share any pretrained models for direct inference？

Hi, Exist a possibility about up this repo to https://replicate.com/ for best use? or Can I up it?

I tried to train and test with your code, but it reported an error. It feels like the model code for training and testing is different.

Hello,
I tried running 'main_test_fbcnn_color.py' on a real JPEG image using one 16 GB V100 but the code threw ´Out of Memory´ error. Any idea on how to use this code with large images, say 12 MPix or more?

Hello @jiaxi-jiang, Sorry to bother you,

I test some non-photo content jpeg(2d drawing),
high FBCNN QF can preserve detail and noise,
but I notice some area have slight color(chroma) brightness change.

In this case, FBCNN QF 70 have slight brightness change on dark red area(red circle),
could you teach me how to improve color accurate for non-photo content?

original image (jpeg q75 420),

png 8 bit depth (FBCNN QF 70)

Other sample(QF 30,50,70)

sample.zip

Hi, It is a nice job to enable flexible QF embedding in JPEG deblocking!
I want to re-produce the results following the directions in the paper, However, there are some questions.

1. qf_loss
   Following the paper, I calculate the loss as:

            input = batch["degree"].type(Tensor)
            label = batch["label"].type(Tensor)
            qf_label = batch["qf"].type(Tensor)
            qf_label = (1-qf_label/100) # 0-1
            out, qf_pred = model(input)

            mse = mse_pixelwise(out, label)

            cls_loss = l1_pixelwise2(qf_pred, qf_label)

            loss = l1_pixelwise(out, label) * 0.5  + mse * 0.5 + cls_loss * 0.1

However, during the training phase, the cls_loss is not changed at all

2021-10-09 22:38:47,481 - __main__ - INFO - 
[QF 10 lr 0.000100 cls_loss:0.23448  loss:0.04852 Epoch 0/120] [Batch 19/203] [psnr: 19.300194 ]  ETA: 6:18:37.362892   [mse: 0.005858 ] [mse_original: 0.000583 ]
2021-10-09 22:39:08,319 - __main__ - INFO - 
[QF 10 lr 0.000100 cls_loss:0.20090  loss:0.03645 Epoch 0/120] [Batch 39/203] [psnr: 21.449274 ]  ETA: 7:02:01.486693   [mse: 0.003679 ] [mse_original: 0.000384 ]
2021-10-09 22:39:29,365 - __main__ - INFO - 

2021-10-10 05:11:26,158 - __main__ - INFO - 
[QF 10 lr 0.000020 cls_loss:0.20545  loss:0.02607 Epoch 119/120] [Batch 159/203] [psnr: 34.804854 ]  ETA: 0:00:41.660994   [mse: 0.000345 ] [mse_original: 0.000475 ]
2021-10-10 05:11:45,303 - __main__ - INFO - 
[QF 10 lr 0.000020 cls_loss:0.22516  loss:0.02851 Epoch 119/120] [Batch 179/203] [psnr: 34.770472 ]  ETA: 0:00:22.570621   [mse: 0.000385 ] [mse_original: 0.000513 ]
2021-10-10 05:12:04,498 - __main__ - INFO - 
[QF 10 lr 0.000020 cls_loss:0.18704  loss:0.02385 Epoch 119/120] [Batch 199/203] [psnr: 34.775089 ]  ETA: 0:00:03.771435   [mse: 0.000276 ] [mse_original: 0.000377 ]
2021-10-10 05:12:12,970 - __main__ - INFO - QF: 10 [PSNR: live1: 29.005289    classic: 29.159697]  
 [SSIM: live1: 0.806543    classic: 0.795164] 
 [max PSNR in live1: 29.005289, max epoch: 119]

The cls_loss is always at 2, and when I test the qf estimation with input image in qf=[10,20,..,90], the qf estimation is always 53,

which shows that the cls_loss does not work well. It is reasonable that qf estimation is 50 since it is good for L1 loss when not trained well.

Due to the GPU limitation, I set the Batchsize = 96, and training with DIV2K patches for about 2.5W iters

Is there anything wrong in my implementation, thanks!

AssertionError: ../trainsets/Data_DIV2K_Flickr2K is not a valid directory During the training process, a question arises, what is the architecture distribution of the training dataset?

hello, it seems the results of the real dataset are not run from the fbcnn_color.pth ? Would you mind providing the corresponding fbcnn model for the real dataset?
Thank you!

run default code, show me error:
model.load_state_dict(torch.load(model_path), strict=True):
error in loading state_stict for FBCNN:
Missing keys in state_stict :
......

but when run fbcnn_color.py , it is ok.
why ?

Hi,
Great works! I test your model(fbcnn_color.pth) in 'testset/Real' dataset. The results are not so remarkable as the picture in this master. The output of model(fbcnn_color.pth) without qf_input are as follow(left is input, right is output):

I don't kown if there are something wrong with my results. And the output of model(fbcnn_color.pth) with qf_input are also not so good. When zoom out, I can find obvious artifacts. Hope for your reply.

Hello @jiaxi-jiang, Sorry to bother you,

I plan apply FBCNN for my non-photo content jpeg(2d drawing),
but I have some question about tool implement,
could you teach me about those question?

1. I have some large jpeg(4441x6213) want apply FBCNN,
   but I don't have enough RAM to process this image,
   probably can implement tile function for FBCNN?

I find a split_imageset function in FBCNN utils_image.py,
I try to invoke this function,
but I haven't find a good method to implement split and merge function.

2. I notice FBCNN output is 8 bit depth png,
   probably implement output(input) 16 bit depth png function, can get better result?

Thank you produce this amazing tool. :)