amadeusitgroup / moire-pattern-detection Goto Github PK

License: Other

Jupyter Notebook 85.10% Python 14.90%

moire-pattern wavelet-decomposition cnn python keras computer-vision

moire-pattern-detection's Introduction

Moire Pattern Detection

Introduction

To detect Moire ́ patterns, images are first decomposed using Wavelet decomposition and trained using multi-input Convolutional neural network. The strength of the proposed CNN model is, it uses the LL intensity image (from the Wavelet decomposition) as a weight parameter for the Moire ́ pattern, thereby approximating the spatial spread of the Moire ́ pattern in the image. Usage of CNN model performs better than frequency thresholding approach as the model is trained considering diverse scenarios and it is able to distinguish between the high frequency of background texture and the Moire ́ pattern.

If this code helps with your work, please cite:

@INPROCEEDINGS{8628746,
author={E. {Abraham}},
booktitle={2018 IEEE Symposium Series on Computational Intelligence (SSCI)},
title={Moiré Pattern Detection using Wavelet Decomposition and Convolutional Neural Network},
year={2018},
volume={},
number={},
pages={1275-1279},
ISSN={},
month={Nov},}

Set-up

1. Install Python3

2. Install dependencies

pip install tensorflow

pip install keras

pip install Pillow

pip install scikit-learn

pip install scikit-image

pip install PyWavelets

Note - Resize Images to WIDTH = 1000 and HEIGHT = 750

3. create the wavelet decomposed training dataset from the captured images

python createTrainingData.py positiveImages negativeImages train

positional arguments:

  positiveImages  Directory with positive (Moiré pattern) images.
  
  negativeImages  Directory with negative (Normal) images.
  
  train           0 = train, 1 = test

4. train the CNN model using training images

python train.py positiveImages negativeImages trainingDataPositive trainingDataNegative epochs

positional arguments:

  positiveImages        Directory with original positive (Moiré pattern)
                        images.
                        
  negativeImages        Directory with original negative (Normal) images.
  
  trainingDataPositive  Directory with transformed positive (Moiré pattern)
                        images.
                        
  trainingDataNegative  Directory with transformed negative (Normal) images.
  
  epochs                Number of epochs for training

5. test the CNN model using test images

python test.py moirePattern3CNN_.h5 positiveImages negativeImages

positional arguments:

  weightsFile               saved CNN model file
  
  positiveTestImages        Directory with original positive (Moiré pattern)
                        images.
                        
  negativeTestImages        Directory with original negative (Normal) images.

Documentation

IEEE SSCI-2018 publication (https://ieeexplore.ieee.org/document/8628746).

Note: The paper shows 3 bands of wavelet decomposition images taken as input to CNN as compared to the 4 bands in this python implementation

Additional information

If you have any technical questions, feel free to contact us or create an issue here.

moire-pattern-detection's People

Contributors

Stargazers

Watchers

moire-pattern-detection's Issues

Batch training of the dataset

Problem: Current implementation takes bulk dataset to train the model. If the dataset is huge, memory availability becomes a problem

Solution: Implement batch training.
Few pointers to start -

Inconsistency between what is told and what is written.

In the train.py script there's the following piece of code:

# read all images, convert to float, divide by 255 (leads to gray range 0..1), reshape into a row vector
# write class 1 for positive and 0 for negative samples    
sampleIndex = readImageSet(positiveImageFiles, positiveTrainImagePath, X_LL, X_LH, X_HL, X_HH, X_index, Y, sampleIndex, 0)
print('positive data loaded.')

sampleIndex += readImageSet(negativeImageFiles, negativeTrainImagePath, X_LL, X_LH, X_HL, X_HH, X_index, Y, sampleIndex, 1)
print('negative data loaded.')

Then there's 2 point of inconsistencies:

Actually the values of the images are not divided by 255, but rescaled by MinMaxScaler.
- It is slightly different, but this way it is dependent of the values present on the images of the dataset.
- Is this side effect desired?
The comment says that class 1 is for the positive samples and class 0 is for the negative samples.
- This makes sense, but it's implemented inversed, that is, 0 for positives and 1 for negatives.
- I think that this may lead to the same results, since it is a binary classification. But I would like to know if there was a reason for this change or was it just a misunderstood?

Thank you in advance for the attention.

Fail to run without GPU?

I followed the instruction and failed on the testing session on a computer without GPU.

Following is the error info. The fatal error seems to be "Shape mismatch: The variable shape (32, 2), and the assigned value shape (32, 1) are incompatible." But I don't know whether it has to do with the GPU.

2022-08-02 14:42:14.891607: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-08-02 14:42:14.891711: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
positive samples: 3
negative samples: 4
training image rotated
training image rotated
training image rotated
Total positive files after augmentation: 9
Total negative files after augmentation: 12
positive samples: 12
negative samples: 16
positive data loaded.
negative data loaded.
Total Samples Loaded: 30
[[0.77935225 0.78643727 0.78036439 ... 0.11032388 0.11234818 0.10931174]
[0.10931174 0.11234818 0.11032388 ... 0.78036439 0.78643727 0.77935225]
[0.50303644 0.71457487 0.69939274 ... 0.5688259 0.57085019 0.57388663]
...
[0. 0. 0. ... 0. 0. 0. ]
[0. 0. 0. ... 0. 0. 0. ]
[0. 0. 0. ... 0. 0. 0. ]]
[[-0.01953819 -0.0017762 -0.0017762 ... -0.0017762 -0.00888099
0.008881 ]
[-0.00888094 0.00888105 0.00177626 ... 0.00177626 0.00177626
0.01953825]
[-0.34991118 0.03730018 -0.03019538 ... 0.0017762 -0.01243339
-0.0017762 ]
...
[ 0. 0. 0. ... 0. 0.
0. ]
[ 0. 0. 0. ... 0. 0.
0. ]
[ 0. 0. 0. ... 0. 0.
0. ]]
[[0.]
[0.]
[0.]
[0.]
[0.]
[0.]
[0.]
[0.]
[0.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[1.]
[0.]
[0.]
[0.]
[0.]
[0.]
[0.]
[0.]]
2022-08-02 14:42:34.738089: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2022-08-02 14:42:34.738195: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-08-02 14:42:34.750206: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: IMASONLIU-PC5
2022-08-02 14:42:34.750471: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: IMASONLIU-PC5
2022-08-02 14:42:34.751354: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Traceback (most recent call last):
File "C:\Users\Administrator\Desktop\Moire-Pattern-Detection-master\Moire-Pattern-Detection-master\src\test.py", line 66, in
main(parse_arguments(sys.argv[1:]))
File "C:\Users\Administrator\Desktop\Moire-Pattern-Detection-master\Moire-Pattern-Detection-master\src\test.py", line 45, in main
CNN_model.load_weights(weights_file)
File "C:\Users\Administrator\anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "C:\Users\Administrator\anaconda3\lib\site-packages\keras\backend.py", line 4028, in batch_set_value
x.assign(np.asarray(value, dtype=dtype_numpy(x)))
ValueError: Cannot assign value to variable ' dense_1/kernel:0': Shape mismatch.The variable shape (32, 2), and the assigned value shape (32, 1) are incompatible.

training error

Traceback (most recent call last):
File "train.py", line 321, in
main(parse_arguments(sys.argv[1:]))
File "train.py", line 44, in main
X_LL, X_LH, X_HL, X_HH, X_index, Y, imageCount = readWaveletData(positiveImagePath, negativeImagePath, positiveTrainImagePath, negativeTrainImagePath)
File "train.py", line 140, in readWaveletData
sampleIndex = readImageSet(positiveImageFiles, positiveTrainImagePath, X_LL, X_LH, X_HL, X_HH, X_index, Y, sampleIndex, 0)
File "train.py", line 99, in readImageSet
ret = readAndScaleImage(f, '', trainImagePath, X_LL, X_LH, X_HL, X_HH, X_index, Y, sampleIndex, bClass)
File "train.py", line 82, in readAndScaleImage
imgVector = imgLL.reshape(1, WIDTH*HEIGHT)
ValueError: cannot reshape array of size 120000 into shape (1,187500)

thanks.

how about the accuracy of your model?

Can you share the moirePattern3CNN_.h5

Hi
Thanks for your excellent code, I want to test code on test image,
can you share the H5 file?
my email is [email protected]

Thanks

Tensorflow error

When executing

python3 train.py positiveImages negativeImages trainDataPositive trainDataNegative 2

(Note that in your README you have trainingDataPositive and trainingDataNegative),

I get the following error:

tensorflow.python.framework.errors_impl.InvalidArgumentError: Received a label value of 1 which is outside the valid range of [0, 1). Label values: 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

I needed to use binary_crossentropy a la https://stackoverflow.com/a/44152923/1021819

Training Error

Even after scaling images to WIDTH, HEIGHT, Training is not possible. Keeps showing sam error.

Could the author provide the download address of the dataset or datasets conveniently?

I would be grateful if you could provide.

Code error

1.Data is augmented by three times, so line 123 of the train.py should be

positiveCount = len(positiveImageFiles)*3
negativeCount = len(negativeImageFiles)*3

2.The amount of FN and FP is reversed.

Some places are inconsistent with the paper

The original paper uses three channels, and your code uses four channels. In the original paper, C2, C3 and C4 use 16 kernels of size 3 x 3 with a stride of 1 pixel, and your C5 and C6 use 32 kernels. The paper uses one dropopt, and your code uses three.
Will the effect of these changes be better than that of the original paper?

training dataset

requirements

hello,can you share the packages version? thx

Do you have the moiré pattern image generator code?

Cannot replicate result of Haar transform

In the playground.ipynb notebook, I cannot replicate the first block of code as good as the original notebook image.

This is my output. As you can see there is not much Moire pattern in here. Especially the HL part

I don't see any changes to this file or haar2D.py on GitHub's history, and you implemented the transform by numpy so I don't think it's a version issue. Is there any explanation to how this is happening?

predict image issue

I did training and got dataset .h5
How i can predict for single/multi image to get result as positive(0) or negative(1).

please support to share/add the file for prediction image.

thanks in advance

predict error

aleImage
imgVector = imgLL.reshape(1, WIDTH*HEIGHT)
ValueError: cannot reshape array of size 126025 into shape (1,187500)

Can you share your model ?

Can you share your model (moirePattern3CNN_.h5) to email [email protected]. Thanks!

Problems during training

Hello! Thanks for sharing！I encountered some problems during fitting CNN model。
Hope for your help！
The following is the training log：
1/102 [..............................] - ETA: 19:24 - loss: 0.6917 - accuracy: 0.5938
2/102 [..............................] - ETA: 8:29 - loss: 0.6912 - accuracy: 0.5938
3/102 [..............................] - ETA: 8:18 - loss: 0.6808 - accuracy: 0.6042
4/102 [>.............................] - ETA: 8:10 - loss: 0.6791 - accuracy: 0.5859
5/102 [>.............................] - ETA: 8:07 - loss: 0.6739 - accuracy: 0.6062
6/102 [>.............................] - ETA: 8:00 - loss: 0.6623 - accuracy: 0.6146
7/102 [=>............................] - ETA: 7:56 - loss: 0.6399 - accuracy: 0.6339
8/102 [=>............................] - ETA: 7:51 - loss: 0.6412 - accuracy: 0.6328
9/102 [=>............................] - ETA: 7:46 - loss: 0.6255 - accuracy: 0.6493
10/102 [=>............................] - ETA: 7:40 - loss: 0.6102 - accuracy: 0.6656
11/102 [==>...........................] - ETA: 7:36 - loss: 0.6136 - accuracy: 0.6619
12/102 [==>...........................] - ETA: 7:30 - loss: 0.6283 - accuracy: 0.6589
13/102 [==>...........................] - ETA: 7:25 - loss: 0.6245 - accuracy: 0.6635
14/102 [===>..........................] - ETA: 7:20 - loss: 0.6191 - accuracy: 0.6652
15/102 [===>..........................] - ETA: 7:15 - loss: 0.6211 - accuracy: 0.6708
16/102 [===>..........................] - ETA: 7:09 - loss: 0.6234 - accuracy: 0.6699
17/102 [====>.........................] - ETA: 7:03 - loss: 0.6240 - accuracy: 0.6728
18/102 [====>.........................] - ETA: 6:58 - loss: 0.6233 - accuracy: 0.6753
19/102 [====>.........................] - ETA: 6:53 - loss: 0.6240 - accuracy: 0.6694
20/102 [====>.........................] - ETA: 6:47 - loss: 0.6217 - accuracy: 0.6656
21/102 [=====>........................] - ETA: 6:42 - loss: 0.6227 - accuracy: 0.6652
22/102 [=====>........................] - ETA: 6:37 - loss: 0.6223 - accuracy: 0.6605
23/102 [=====>........................] - ETA: 6:32 - loss: 0.6200 - accuracy: 0.6603
24/102 [======>.......................] - ETA: 6:27 - loss: 0.6188 - accuracy: 0.6576
25/102 [======>.......................] - ETA: 6:22 - loss: nan - accuracy: 0.6637
26/102 [======>.......................] - ETA: 6:16 - loss: nan - accuracy: 0.6623
27/102 [======>.......................] - ETA: 6:11 - loss: nan - accuracy: 0.6562
28/102 [=======>......................] - ETA: 6:06 - loss: nan - accuracy: 0.6607
29/102 [=======>......................] - ETA: 6:00 - loss: nan - accuracy: 0.6595
30/102 [=======>......................] - ETA: 5:55 - loss: nan - accuracy: 0.6583
31/102 [========>.....................] - ETA: 5:50 - loss: nan - accuracy: 0.6562
32/102 [========>.....................] - ETA: 5:45 - loss: nan - accuracy: 0.6562
33/102 [========>.....................] - ETA: 5:39 - loss: nan - accuracy: 0.6544
34/102 [=========>....................] - ETA: 5:34 - loss: nan - accuracy: 0.6535
35/102 [=========>....................] - ETA: 5:29 - loss: nan - accuracy: 0.6509
36/102 [=========>....................] - ETA: 5:24 - loss: nan - accuracy: 0.6432
37/102 [=========>....................] - ETA: 5:19 - loss: nan - accuracy: 0.6385
38/102 [==========>...................] - ETA: 5:14 - loss: nan - accuracy: 0.6398
39/102 [==========>...................] - ETA: 5:09 - loss: nan - accuracy: 0.6410
40/102 [==========>...................] - ETA: 5:04 - loss: nan - accuracy: 0.6398
41/102 [===========>..................] - ETA: 4:59 - loss: nan - accuracy: 0.6418
42/102 [===========>..................] - ETA: 4:54 - loss: nan - accuracy: 0.6384
43/102 [===========>..................] - ETA: 4:49 - loss: nan - accuracy: 0.6344
44/102 [===========>..................] - ETA: 4:44 - loss: nan - accuracy: 0.6385
45/102 [============>.................] - ETA: 4:39 - loss: nan - accuracy: 0.6396
46/102 [============>.................] - ETA: 4:34 - loss: nan - accuracy: 0.6372
47/102 [============>.................] - ETA: 4:29 - loss: nan - accuracy: 0.6343

98/102 [===========================>..] - ETA: 19s - loss: nan - accuracy: 0.6279
99/102 [============================>.] - ETA: 14s - loss: nan - accuracy: 0.6294
100/102 [============================>.] - ETA: 9s - loss: nan - accuracy: 0.6309
101/102 [============================>.] - ETA: 4s - loss: nan - accuracy: 0.6315
102/102 [==============================] - ETA: 0s - loss: nan - accuracy: 0.6315
Epoch 1: val_loss did not improve from inf

102/102 [==============================] - 519s 5s/step - loss: nan - accuracy: 0.6315 - val_loss: nan - val_accuracy: 0.5861
Why does the loss became nan after the 24th iteration, and what means 'val_loss did not improve from inf' ?
Did I do something wrong during training or data preparation?
Hope for your help！

>python train.py positiveImages negativeImages trainingDataPositive trainingDataNegative 30    
positive samples: 12
negative samples: 16                                                                                 
Error: Couldnt read the file 284_letterbox1024. Make sure only images are present in the folder      
Exception: [Errno 2] No such file or directory: 'trainingDataPositive\\284_letterbox1024_LL.tiff'    
Error: Couldnt read the file 284_letterbox1024. Make sure only images are present in the folder      
Exception: [Errno 2] No such file or directory: 'trainingDataPositive\\284_letterbox1024_180_LL.tiff'
Error: Couldnt read the file 284_letterbox1024. Make sure only images are present in the folder      
...

> python test.py moirePattern3CNN_.h5 positiveImages negativeImages
...
Total Samples Loaded:  30
...
ValueError: Cannot assign value to variable ' dense_1/kernel:0': Shape mismatch. The variable shape (32, 2), and the assigned value shape (32, 1) are incompatible.

I modified the code and the error was dismissed. The "After" code is right?

Before

python createTrainingData.py positiveImages negativeImages 0
python train.py positiveImages negativeImages trainingDataPositive trainingDataNegative 5
python test.py moirePattern3CNN_.h5 positiveImages negativeImages

After

python createTrainingData.py positiveImages negativeImages 0
python train.py positiveImages negativeImages trainDataPositive trainDataNegative 5
python test.py moirePattern3CNN_.h5 positiveImages negativeImages