Code Monkey home page Code Monkey logo

dsc's Introduction

Direction-Aware Spatial Context Features for Shadow Detection (and Removal)

by Xiaowei Hu, Chi-Wing Fu, Lei Zhu, Jing Qin and Pheng-Ann Heng

This implementation is written by Xiaowei Hu at the Chinese University of Hong Kong.


PyTorch Version

A PyTorch version is available at https://github.com/stevewongv/DSC-PyTorch implemented by Tianyu Wang.

Results

The shadow detection results on the SBU and UCF datasets can be found at Google Drive.
The shadow detection results on the new split of UCF (used by some works) can be found at Google Drive; BER: 10.38, accuracy: 0.95.

The shadow removal results on the SRD and ISTD datasets can be found at Google Drive.

Citation

@InProceedings{Hu_2018_CVPR,
     author = {Hu, Xiaowei and Zhu, Lei and Fu, Chi-Wing and Qin, Jing and Heng, Pheng-Ann},
     title = {Direction-Aware Spatial Context Features for Shadow Detection},
     booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
     pages={7454--7462},
     year = {2018} }

@article{hu2020direction,
     author = {Hu, Xiaowei and Fu, Chi-Wing and Zhu, Lei and Qin, Jing and Heng, Pheng-Ann},
     title = {Direction-Aware Spatial Context Features for Shadow Detection and Removal},
     journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
     volume={42},
     number={11},
     pages={2795--2808},
     year = {2020}
}

Installation

  1. Please download and compile our CF-Caffe.

  2. Clone the DSC repository, and we'll call the directory that you cloned as DSC-master.

    git clone https://github.com/xw-hu/DSC.git
  3. Replace CF-Caffe/examples/ by DSC-master/examples/. Replace CF-Caffe/data/ by DSC-master/data/.

Test

Shadow Detection

  1. Please download our pretrained model at Google Drive.
    Put this model in examples/DSC/DSC_detection/snapshot/.

  2. (Matlab User) Enter the examples/DSC/ and run test_detection.m in Matlab.

  3. (Python User) Enter the examples/DSC/DSC_detection/ and export PYTHONPATH in the command window such as:

    export PYTHONPATH='../../../python'

    Run the test model and resize the results to the size of original images:

    ipython notebook DSC_test.ipynb
  4. Apply CRF to do the post-processing for each image.
    The code for CRF can be found in https://github.com/Andrew-Qibin/dss_crf
    *Note that please provide a link to the original code as a footnote or a citation if you plan to use it.

Shadow Removal

Enter the examples/DSC/ and run test_removal.m in Matlab.

Train

Download the pre-trained VGG16 model at http://www.robots.ox.ac.uk/~vgg/research/very_deep/.
Put this model in CF-Caffe/models/

Shadow Detection

  1. Enter the examples/DSC/DSC_detection/
    Modify the image path in DSC.prototxt.

  2. Run

    sh train.sh

Shadow Removal

  1. Color compensation mechanism:
    Enter the /data/SRD/ or /data/ISTD/.
    Run color_transfer_function.m in Matlab.

  2. Transfer the images into the LAB color sapce and do the data argumentation:
    Enter the /data/SRD/ or /data/ISTD/.
    Run ToLab.m and data_argument.m in Matlab.

  3. Enter the examples/DSC/DSC_removal_SRD/ or examples/DSC/DSC_removal_ISTD/.
    Modify the image path in DSC.prototxt.

  4. Run

    sh train.sh

dsc's People

Contributors

xw-hu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dsc's Issues

Test file error

Hi,
I have build the DSC according to your instruction and build successfully. but when i want to build the test I got the following error..
channel_wise_lyer

Can you help in this regard.

Thanks

模型求助

你好,pre-trained model,ResNeXt101 model国内可以下载吗,没有VPN[手动笑哭][手动笑哭]

loss Function

请问胡老师,你的损失函数有pytorch版本吗?

During running DSC_test.ipynb, "caffe.LayerParameter" has no field named "irnn_param".

During running DSC_test.ipynb, "caffe.LayerParameter" has no field named "irnn_param".

My Spec/Environment:
anaconda, caffe-gpu on Ubuntu18.04
Caffe was installed by Conda install.

On running
net = caffe.Net('deploy.prototxt', 'snapshot/DSC_iter_12000.caffemodel', caffe.TEST)

Error below:

WARNING: Logging before InitGoogleLogging() is written to STDERR
W0329 21:54:29.432673  3982 _caffe.cpp:139] DEPRECATION WARNING - deprecated use of Python interface
W0329 21:54:29.432858  3982 _caffe.cpp:140] Use this instead (with the named "weights" parameter):
W0329 21:54:29.432874  3982 _caffe.cpp:142] Net('/home/einstein/DSC/examples/DSC/DSC_detection/deploy.prototxt', 1, weights='/home/einstein/DSC/examples/DSC/DSC_detection/snapshot/DSC_iter_12000.caffemodel')
[libprotobuf ERROR google/protobuf/text_format.cc:307] Error parsing text-format caffe.NetParameter: 524:15: Message type "caffe.LayerParameter" has no field named "irnn_param".
F0329 21:54:29.437965  3982 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /home/einstein/DSC/examples/DSC/DSC_detection/deploy.prototxt
*** Check failure stack trace: ***

Thanks.

Error in Running the Testing Code in Matlab as well python

Hi,
I download the your trained model from the google drive and save it into models folder in the DSC root folder. Change the required paths in matlab code for depoly and model files and try to run the code in matlab but i got the following error. Can you help me in this regards,
matlab_error

After this i try to run the python Code and write it into python file as attached and than try to run the code but this time i got the following error.
python_error

Help me how to run the testing code for detection.

About the shadow removal evaluation of SRD dataset

Hello,

wonderful job I think.

When evaluating the shadow removal results of SRD, my result is 16.83 RMSE for the whole image. I simply use the binarization of the difference of shadow free and shadow images. Could you share me the training and testing shadow mask you used? Thanks in advance.

训练报错

老师您好,我想问一下我训练时出现以下错误导致不能训练成功,可以帮我解答以下为什么吗?
I0411 18:55:27.986623 13702 layer_factory.hpp:77] Creating layer data
I0411 18:55:27.986646 13702 net.cpp:100] Creating Layer data
I0411 18:55:27.986651 13702 net.cpp:408] data -> data
I0411 18:55:27.986665 13702 net.cpp:408] data -> label
I0411 18:55:27.986709 13702 image_labelmap_data_layer.cpp:41] Opening file CF-Caffe/data/SBU/train.txt
I0411 18:55:27.986716 13702 image_labelmap_data_layer.cpp:51] Shuffling data
I0411 18:55:27.986719 13702 image_labelmap_data_layer.cpp:56] A total of 0 images.
*** Aborted at 1554980127 (unix time) try "date -d @1554980127" if you are using GNU date ***
PC: @ 0x7fadc51aa7a3 std::operator+<>()
*** SIGSEGV (@0x8) received by PID 13702 (TID 0x7fadc5e49200) from PID 8; stack trace: ***
@ 0x7fadc389ef20 (unknown)
@ 0x7fadc51aa7a3 std::operator+<>()
@ 0x7fadc5269571 caffe::ImageLabelmapDataLayer<>::DataLayerSetUp()
@ 0x7fadc522e569 caffe::BasePrefetchingLabelmapDataLayer<>::LayerSetUp()
@ 0x7fadc5111483 caffe::Net<>::Init()
@ 0x7fadc5112ab1 caffe::Net<>::Net()
@ 0x7fadc5137c1e caffe::Solver<>::InitTrainNet()
@ 0x7fadc5138286 caffe::Solver<>::Init()
@ 0x7fadc513860a caffe::Solver<>::Solver()
@ 0x7fadc51803b3 caffe::Creator_SGDSolver<>()
@ 0x557c537d64bd train()
@ 0x557c537d257a main
@ 0x7fadc3881b97 __libc_start_main
@ 0x557c537d2f7a _start
Segmentation fault (core dumped)
我的数据集建立在一个train文件夹下,里面同时放入shadowimages和shadowmasks,再更该路径,不知是否正确。

The size of the test set on SRD is 405?Shouldn't it be 408?

Hello, your work is very exciting, thank you for your contribution to this field.
I have some doubts about the test results.
I download the result for shadow removal on SRD.
image
The size of the test set on SRD is 405. Shouldn't it be 408?
Thank you for your work and look forward to your reply.

配置好caffe之后,在跑detection.m程序的时候,matlab闪退

配置好caffe之后,也用小猫的那个test验证过,成功了。但是在跑这个test_detection这个程序的时候,matlab闪退了,我用的matlab版本是2016a,也用2017a试过,报错说:“MATLAB has encountered an internal problem and needs to close.”

求助!

胡老师,您好!我在网络上看到您的论文,下载使用此程序,但是一直提示不成功,您能否提供一个更为详细的帮助文档或者在线协助一下,非常感谢。qq:505857067
image

NO model named DSC_iter_160000_lab.caffemodel is found

Hello, I am interested in your work about shadow removing and I have an issue. In the file examples/DSC/test_removal.m, there is an assertion for the existence of the file 'snapshot/DSC_iter_160000_lab.caffemodel'. I guess it is the model trained by your lab to remove the shadow, however I can't find it anywhere or find any access to it in your description file README.md, so could you please tell where can I find it? Thanks a lot

BER和accuracy

老师你好,很抱歉打扰您啦,请问能分享一下BER和accuracy计算的python代码吗,我看过torch版本的,但是好像没有涉及到这两个评价指标,或者是我没有发现,期待您的回复!谢谢~

Out of Memory Error

Hi
I am using your code and want to retrain the model on my own data. but i got out of memory error. Following is the error and GPU specification. Can you you tell me how much memory is required for training ?
dsc_training_error

GPU Specification
nvidia_specification

irnn-layer missing

I am trying to run the shadow detection but when inferring the net in python an error occurs. The error occurs at net.forward(). I compiled the CF-Caffe and no error occured during the compilation.

error looks like following:
F0326 13:43:25.415571 15771 irnn_layer-cpp:73] Not Implemented Yet
*** Check failure stack trace: ***
Aborted (core dumped)

Does someone have an idea how to solve this problem?

Thank you in advance!

loss is so big

Hi, when I run train.py in DSC_detection, the loss was very big, like this:
Iteration 60, loss = 640045 I1116 18:20:46.603199 22386 solver.cpp:244] Train net output #0: loss-dsn1 = 122860 (* 1 = 122860 loss) I1116 18:20:46.603204 22386 solver.cpp:244] Train net output #1: loss-dsn2 = 117984 (* 1 = 117984 loss) I1116 18:20:46.603207 22386 solver.cpp:244] Train net output #2: loss-dsn3 = 123544 (* 1 = 123544 loss) I1116 18:20:46.603211 22386 solver.cpp:244] Train net output #3: loss-dsn4 = 84440.4 (* 1 = 84440.4 loss) I1116 18:20:46.603215 22386 solver.cpp:244] Train net output #4: loss-dsn5 = 104531 (* 1 = 104531 loss) I1116 18:20:46.603219 22386 solver.cpp:244] Train net output #5: loss-dsn6 = 87796.2 (* 1 = 87796.2 loss) I1116 18:20:46.603222 22386 solver.cpp:244] Train net output #6: loss-fuse = 79751.9 (* 1 = 79751.9 loss) I1116 18:20:46.603225 22386 solver.cpp:244] Train net output #7: loss-global = 120688 (* 1 = 120688 loss)
Could you give me some suggestions? Thank you very much!

Which part reflect 'Direction-aware attention mechanism' in the code?

Hello, Mr. Hu
I hope to introduce it 'Direction-aware attention mechanism' into my experiment.I hope you can tell me where it is in the code.Is it in the DSC/examples/DSC/DSC_detection/DSC.prototxt?
Also, I would like to ask whether DSC module can be used in other parts besides focusing on the shadow part. For example, in high exposure areas, if possible, which parameters determine?
I look forward to your reply. Thank you.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.