wangchx67 / fourllie Goto Github PK

This is the official pytorch implementation of "FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information" (ACM MM 2023)

License: MIT License

Python 100.00%

fourllie's Introduction

FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information

Introduction

This is the official pytorch implementation of "FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information" (ACM MM 2023)

We design a two-stage framework to enhance low-light images with the help of the Fourier frequency information. In the first stage, we improve the lightness of low-light images by estimating the amplitude transform map in the Fourier space. In the second stage, we introduce the Signal-to-Noise-Ratio (SNR) map to provide the prior for integrating the global Fourier frequency and the local spatial information, which recovers image details in the spatial space. With this ingenious design, FourLLIE outperforms the existing state-of-the-art (SOTA) LLIE methods on four representative datasets while maintaining good model efficiency.

Installation

conda create --name FourLLIE --file requirements.txt
conda activate FourLLIE

Train

You can modify the training configuration (e.g., the path of datasets, learning rate, or model settings) in ./options/train/LOLv2_real.yml and run:

python train.py -opt ./options/train/LOLv2_real.yml

Test

Modify the testing configuration in ./options/test/LOLv2_real.yml and run:

python test.py -opt ./options/test/LOLv2_real.yml

Datasets

LOL-real and LOL-sys can be found in here.
LSRW-Huawei and LSRW-Nikon can be found in here.

Pre-trained

We update the pre-trained model in the ./pre-trained. Note that the initial models of LSRW is missing and we re-trained them. The metric results may not exactly match the results reported in paper. You can freely choose any version.

Acknowledgement

This repo is based on SNR-Aware.

Citation Information

If you find the project useful, please cite:

@inproceedings{wang2023fourllie,
  title={FourLLIE: Boosting Low-Light Image Enhancement by Fourier Frequency Information},
  author={Wang, Chenxi and Wu, Hongjun and Jin, Zhi},
  booktitle={Proceedings of the 31st ACM International Conference on Multimedia},
  pages={7459--7469},
  year={2023}
}

fourllie's People

Contributors

Stargazers

Watchers

Forkers

dkfmpdgv ip-enhancement peterpaniff rrongzz sydcs davorjordacevic

fourllie's Issues

About inference time

Hi, thanks for your contribution！

When debugging FourLLIE, I was wondering why the checkpoint is so small but the inference time is relatively large, e.g., 0.06 seconds on 3090 GPU. Is there anyway to accelerate your model？

Thanks

该算法在windows上可以运行吗？

t-SNE 的图

同学，你好，我想咨询一下，您的Figure 1中的（b）图代码能提供么，最近赶论文需要借用一下，谢谢~

可以提供一下与训练模型吗？谢谢啦 start，[email protected]

About LOLv2_real configuration

Hello, we use the config file in the folder 'options/train/LOLv2_real.yml' to retrain the model, but the results obtained are a little worse than the ones presented in the paper. I dont know what i did wrong, Can you give me some help? my dataset and config with the results are presented below:

error

您好！我复现时候，指标无法达到论文所示，您可否提供一下您的训练设置，如果有预训练模型将更好，非常感谢！[email protected] 邮箱

您好，可以麻烦您提供一下预训练模型吗[email protected]

配置文件

您好，请问您可以分享一份LOL和LSRW数据集的配置文件吗？

[email protected]

感谢

Missing latest_G.pth

I am getting the following error while reproducing the results.

FileNotFoundError: [Errno 2] No such file or directory: './experiments/train/models/latest_G.pth'

Kindly upload the trained model "latest_G.pth" on your repository or send it to me (email: [email protected]). Thanks

如何可视化Phase和Amplitude分量

您好，请问您方便分量一份关于论文图1(a)的代码吗？可视化Phase和Amplitude分量，并且交换低光照和正常光照的Amplitude之后，再次还原回低光照和正常光照。期待您的回复，十分感谢。邮箱：[email protected]

关于 t-sne

请问能给出生成t-sne文件夹下，那五类图片的脚本文件吗，谢谢！
img_path = "C:/Users\Dell\Desktop\mm23_fourier/tsne\input"

About train

I want to increase the training data and batch size, but an error occurs when I set it to "pytorch". Added environment variable "os.environ['RANK'] = str(0)", still error. So can you provide me with distributed training code "train.py" or what parameters need to be modified? [email protected]

About inference image sizes

Hi, I have a problem when i use the Frequency Stage in your network architecture: I train the model with 256x256 image patches, but i get some strange results when using the full size image (432x624) at test! I was wondering if you encountered the same problem in your experiments and how you solved it

Complexhalf

Excuseme me,when I used the code fft.fft2(),torch.abs(),and torch.angel(),there is a problem "where cuda is not support "complexhalf".So I want to ask how to configure yours envirnoment,and how to solve it? thank you

Pretrained model?

Are you planning to release a trained model? I'm interested to see how this model compares in terms of image quality and inference speed.

如何训练LOLv2_syn数据集？

嗨，感谢您的精彩工作！！我在训练LOLv2-syn数据集时候出现了问题，问题如下所示。然而，我可以成功训练v1和v2-real数据集。能否提供一下LOLv2-syn的yml文件或者指出我存在的问题。十分感谢！！我的邮箱是：[email protected]

Problem about enviroments

When run the code ' conda create --name FourLLIE --file requirements.txt ' encounters an error that some packages can not be found.
So I removed the build pins(everything in environments.txt after the second '=' in each line)
I run conda with default channel, but even python 3.7.12 does not exist as follow.
How did you create this environment?

Could not solve for environment specs
The following packages are incompatible
├─ brotlipy 0.7.0  is installable with the potential options
│  ├─ brotlipy 0.7.0 would require
│  │  └─ python >=3.6,<3.7.0a0  with the potential options
│  │     ├─ python [3.6.10|3.6.2|...|3.8.2] would require
│  │     │  └─ readline [7.* |>=7.0,<8.0a0 ], which can be installed;
│  │     ├─ python [3.10.0|3.10.3|...|3.9.7] would require
│  │     │  └─ libffi >=3.3,<3.4.0a0 , which can be installed;
│  │     └─ python [3.6.10|3.7.7|3.8.2] would require
│  │        └─ libffi >=3.2.1,<3.3a0 , which can be installed;
│  ├─ brotlipy 0.7.0 would require
│  │  └─ python >=3.10,<3.11.0a0  with the potential options
│  │     ├─ python [3.10.0|3.10.3|...|3.9.7], which can be installed (as previously explained);
│  │     ├─ python [3.10.6|3.10.8|...|3.9.15] would require
│  │     │  └─ sqlite >=3.39.3,<4.0a0 , which can be installed;
│  │     ├─ python [3.10.8|3.7.15|3.8.15|3.9.15] would require
│  │     │  └─ sqlite >=3.40.0,<4.0a0 , which can be installed;
│  │     ├─ python [3.10.9|3.11.0|3.7.16|3.8.16|3.9.16] would require
│  │     │  └─ sqlite >=3.40.1,<4.0a0 , which can be installed;
│  │     ├─ python [3.10.11|3.10.12|...|3.9.18] would require
│  │     │  └─ sqlite >=3.41.2,<4.0a0 , which can be installed;
│  │     └─ python [3.10.10|3.11.2] would require
│  │        └─ sqlite >=3.41.1,<4.0a0 , which can be installed;
│  ├─ brotlipy 0.7.0 would require
│  │  └─ python >=3.11,<3.12.0a0 , which can be installed (as previously explained);
│  ├─ brotlipy 0.7.0 would require
│  │  └─ python >=3.7,<3.8.0a0 , which can be installed (as previously explained);
│  ├─ brotlipy 0.7.0 would require
│  │  └─ python >=3.8,<3.9.0a0  with the potential options
│  │     ├─ python [3.6.10|3.6.2|...|3.8.2], which can be installed (as previously explained);
│  │     ├─ python [3.10.0|3.10.3|...|3.9.7], which can be installed (as previously explained);
│  │     ├─ python [3.6.10|3.7.7|3.8.2], which can be installed (as previously explained);
│  │     ├─ python [3.10.6|3.10.8|...|3.9.15], which can be installed (as previously explained);
│  │     ├─ python [3.10.8|3.7.15|3.8.15|3.9.15], which can be installed (as previously explained);
│  │     ├─ python [3.10.9|3.11.0|3.7.16|3.8.16|3.9.16], which can be installed (as previously explained);
│  │     ├─ python 3.8.15 would require
│  │     │  └─ readline >=8.2,<9.0a0 , which can be installed;
│  │     └─ python [3.10.11|3.10.12|...|3.9.18], which can be installed (as previously explained);
│  └─ brotlipy 0.7.0 would require
│     └─ python >=3.9,<3.10.0a0  with the potential options
│        ├─ python [3.10.0|3.10.3|...|3.9.7], which can be installed (as previously explained);
│        ├─ python [3.10.6|3.10.8|...|3.9.15], which can be installed (as previously explained);
│        ├─ python [3.10.8|3.7.15|3.8.15|3.9.15], which can be installed (as previously explained);
│        ├─ python [3.10.9|3.11.0|3.7.16|3.8.16|3.9.16], which can be installed (as previously explained);
│        ├─ python [3.10.11|3.10.12|...|3.9.18], which can be installed (as previously explained);
│        └─ python 3.9.13 would require
│           └─ sqlite >=3.39.2,<4.0a0 , which can be installed;
├─ ffmpeg 4.3  does not exist (perhaps a typo or a missing channel);
├─ libffi 3.4.2  is not installable because it conflicts with any installable versions previously reported;
├─ libgcc-ng 12.2.0  does not exist (perhaps a typo or a missing channel);
├─ libgomp 12.2.0  does not exist (perhaps a typo or a missing channel);
├─ libstdcxx-ng 12.2.0  does not exist (perhaps a typo or a missing channel);
├─ libzlib 1.2.12  does not exist (perhaps a typo or a missing channel);
├─ openssl 3.0.7  does not exist (perhaps a typo or a missing channel);
├─ python 3.7.12  does not exist (perhaps a typo or a missing channel);
├─ pytorch-mutex 1.0  does not exist (perhaps a typo or a missing channel);
├─ pytorch 1.10.0  does not exist (perhaps a typo or a missing channel);
├─ readline 8.1.2  is not installable because it conflicts with any installable versions previously reported;
├─ sqlite 3.38.5  is not installable because it conflicts with any installable versions previously reported;
├─ torchaudio 0.10.0  does not exist (perhaps a typo or a missing channel);
└─ torchvision 0.11.0  does not exist (perhaps a typo or a missing channel).

Discrepancies between the code and the paper

Thank you for your great work, but I still got some problem in the code you supplied.
Why is spatial set to True as illustrate below

It is not concatenated with SPBlock in the paper

Should I set spatial to false in order to be consistent with the paper?