Code Monkey home page Code Monkey logo

mdms's Introduction

【AAAI'2024】Multi-Domain Multi-Scale Diffusion Model for Low-Light Image Enhancement

The official implementation of AAAI24 paper Multi-Domain Multi-Scale Diffusion Model for Low-Light Image Enhancement.

Environment

create a new conda env, and run

$ pip install -r requirements.txt

torch/torchvision with CUDA version >= 11.3 should be fine.

Demo

1. Download pretrained model

Download the Pretrained MDMS model from Baidu NetDisk or Google Drive.

Put the downloaded ckpt in datasets/scratch/LLIE/ckpts.

2. Inference

# in {path_to_this_repo}/,
$ python eval_diffusion.py

Put the test input in datasets/scratch/LLIE/data/lowlight/test/input.

Output results will be saved in results/images/lowlight/lowlight.

Evaluation

Put the test GT in datasets/scratch/LLIE/data/lowlight/test/gt for paired evaluation.

# in {path_to_this_repo}/,
$ python evaluation.py

Results

All results listed in our paper including the compared methods are available in Baidu Netdisk or Google Drive.

  • Note that the provided model is trained on the LOLv1 training set, but generalizes well on other datasets.
  • For SSIM, we directly calculate the performance on RGB channel rather than just grayscale images in PyDiff.
  • For LPIPS, we use a different normalization method (NormA) compared to PyDiff (NormB).

Our method remains superior under the same setting as PyDiff.

1. Test results on LOLv1 test set.

2. Generalization results on LOLv2 syn and real test sets.

3. Generalization results on other unpaired datasets.

We will perform more training and tests on other datasets in the future.

Training

Put the training dataset in datasets/scratch/LLIE/data/lowlight/train.

# in {path_to_this_repo}/,
$ python train_diffusion.py

Detailed training instructions will be updated soon.

Citation

If you find this paper useful, please consider staring this repo and citing our paper:

@inproceedings{shang2024multi,
  title={Multi-Domain Multi-Scale Diffusion Model for Low-Light Image Enhancement},
  author={Shang, Kai and Shao, Mingwen and Wang, Chao and Cheng, Yuanshuo and Wang, Shuigen},
  booktitle={Proceedings of the AAAI Conference on Artificial Intelligence},
  volume={38},
  number={5},
  pages={4722--4730},
  year={2024}
}

mdms's People

Contributors

oli-iver avatar oliiveralien avatar

Stargazers

史恺伦 avatar Shiping avatar  avatar Yuanyang Zhang avatar sk avatar hejh8 avatar Siddharth Shrivastava avatar Wisp avatar  avatar 科研狗 avatar Junhua Liu avatar Xiangpeng Yang avatar Shuai Zhao avatar 周 avatar

Watchers

Kostas Georgiou avatar  avatar

mdms's Issues

关于测试指标

if use_align==True:

现在所有方法都能这么玩吗,推理的时候拿真值进行对齐?如果可以的话,直接按像素点对齐像素值,不是直接满分指标了吗?

About the checkerboard artifacts due to the uneven overlapping

I would like to ask how to understand “uneven overlapping” in the conventional patch-based sampling strategy.
In other tasks, I did not find the checkerboard artifacts shown in the paper in the conventional patch-based sampling strategy. In addition, if the conventional patch-based sampling strategy has checkerboard artifacts, why can the multi-scale method proposed in this paper solve this problem?

测试报错

Traceback (most recent call last):
File "/home/ubuntu/Low-image/duibi/Low-Light/MDMS-main/eval_diffusion.py", line 83, in
main()
File "/home/ubuntu/Low-image/duibi/Low-Light/MDMS-main/eval_diffusion.py", line 79, in main
model.restore(val_loader, validation=args.test_set, r=args.grid_r,use_align=True)
File "/home/ubuntu/Low-image/duibi/Low-Light/MDMS-main/models/restoration.py", line 39, in restore
x_output1 = self.diffusive_restoration(x_cond, r=r,fullresolusion=False)
File "/home/ubuntu/Low-image/duibi/Low-Light/MDMS-main/models/restoration.py", line 80, in diffusive_restoration
x_output = self.diffusion.sample_image(x_cond, x, ii,jj,osize,patch_locs=corners, patch_size=p_size,patch_locs1=corners1,patch_locs2=corners2)
File "/home/ubuntu/Low-image/duibi/Low-Light/MDMS-main/models/ddm.py", line 222, in sample_image
xs = utils.sampling.generalized_steps_overlapping(x, x_cond, seq, self.model, self.betas, ii=ii, jj=jj,
File "/home/ubuntu/Low-image/duibi/Low-Light/MDMS-main/utils/sampling.py", line 99, in generalized_steps_overlapping
et_output[0, :, hi:hi + p_size, wi:wi + p_size] += outputs[idx]
IndexError: index 32 is out of bounds for dimension 0 with size 32

How to validate while training

Hi! I admire your work and I would like to run it on my datasets. But I notice that your released code in DATASETS part set patches for validate "return torch.stack(outputs, dim=0), img_id,ii,jj,size". If I want to validate the whole image, what should I do ? THX!

关于显存

请问你们用的显存是多大?我在文章中Training details没有找到对显卡的说明,我的显存是24G,但是在test的时候,依旧会显存溢出。

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.