lin-shuyu / vae-lstm-for-anomaly-detection Goto Github PK

View Code? Open in Web Editor NEW

402.0 10.0 81.0 2.06 MB

We propose a VAE-LSTM model as an unsupervised learning approach for anomaly detection in time series.

Jupyter Notebook 87.66% Python 12.34%

anomaly-detection unsupervised-learning variational-autoencoder time-series tensorflow

vae-lstm-for-anomaly-detection's People

Contributors

Stargazers

Watchers

Forkers

gyyixr tourir agnes-yang sweetmoment16 cv-ip euler7s dxdezhanghao spokv starflettw wael199 jimmy-inl yasarmajib nicolepcx ducluongtrann hell-to-heaven arita37 liao-com kingdary yiyg510 alheliou ecowmin shliujing machao123343 hsmohammed zhaotianzi mail2surie jinyu25 azm-tarique bbharat3 armlynobinguar heyanyidui wpyx89 ibuprofenighty deborahmit eventhorizon7 crazybirl brightface zillant orientpine overfittingstudyroom krishnaamsk wangxueyuan2020 mm32-star tinghanjxl hisong182 tdl77 cyh4157 kippfreud jiunnguo im1eon aminsn lpldw b0rn2di3 uhhyunjoo limengqigithub qllfq huliqujing hasheng jsyanni zhichaoyou neuerliu yonglee2021 sjdwjt joosm grabias-m aytacpacal lee5xf lunur123 weiwei2202 vwhvpwvk mzhuang1 hajpopey kofiasare266 lewis2222 gopikrishnansrikumar free-angel nguyenxp24 jin-jae jhparkland yiqun-sun-rookie

vae-lstm-for-anomaly-detection's Issues

Question in cudnn, cudatoolkit version

Could you please let me know the cudnn, cudatoolkit version that worked for you?

I have the following configuration,
cudnn : 7.6.0
cudatoolkit : 10.1.243
tensorflow : 1.15.2
However I get the following error when training,
tensorflow/stream_executor/cuda/cuda_blas.cc:428] failed to run cuBLAS routine: CUBLAS_STATUS_EXECUTION_FAILED

I wanted to make sure that my configuration is alright, if not want to align with yours.

Thank you,
Pravin

tensorflow-gpu and tensorflow_probability version issues

tensorflow-gpu == 1.15.2 # Version Matching. Must be version 0.6.1
tensorflow_probability == 0.7.0 # Minimum version 0.8.0
So, what version should i install? tensorflow-gpu choose 1.15.2 or 0.6.1? tensorflow_probability choose 0.7.0 or 0.8.8?

Does this support multi-channel timeseries?

It looks like the code supports multi-channel timeseries, and your paper is formulated generally for the multi-channel case. Do you have an example similar to your NAB example that is a multi-channel?

Thanks

why the the detected sequence add a interval*2 margin when plot the result?

Hello, when I read the code and the finally anomaly detection results figure of your paper, I find the axs.plot( j * interval * 2 ),i cant understand why. If i remove this margin, the detected anomaly will cant cover the true anomaly. So can you explain why? Thanks a lot!

Are the vae and lstm models independent from each other?

Hi,

I have went through your codes and I'm still a little bit confused about the structures of theses two models (I'm not very good at reading codes, forgive me): are they independent from each other?

When I looked at the structure of the vae model, the input is the original slice window at time step t and the output is the reconstructed window at the same time step t, not the window at time step t+1, and there is no layer input or output from the lstm model. So I guess both of models are trained separately, after the vae is trained, the lstm utilizes the information from the vae and predicted the embedding window at the next time step. Is this correct? If this this correct, how to guarantee the convergence of these two models? As the lstm model relies on the performance of the vae model.

Another question is how to process the time series input to the encoder, here I saw you have used convolutional layer, I guess lstm also works?

Example doesn't seem to produce proper result for me

I ran the example in the README.md file and it doesn't look like it produced the correct results. All the reconstructions in the result directory for the experiment are just a constant mean signal (0). Here is what the output looks like from the training. Can you post what a proper result should look like for the example?

Finish processing the embeddings of the entire dataset.
The first a few embeddings are
[[-0.07727078 -0.0207611  -0.1191008   0.15289539  0.06768157  0.00136372]
 [ 0.00486088 -0.00883693  0.00458424  0.02584253  0.01832935 -0.02976965]
 [ 0.00974356  0.00351174 -0.01392651 -0.00980108 -0.00250123 -0.01614434]
 [-0.02508994  0.00390782 -0.07213792  0.06181057  0.02366055 -0.01901692]
 [ 0.00347477  0.00061497 -0.00568073  0.00633631  0.00377738 -0.00562688]]
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 11, 6)]           0         
_________________________________________________________________
lstm (LSTM)                  (None, 11, 64)            18176     
_________________________________________________________________
lstm_1 (LSTM)                (None, 11, 64)            33024     
_________________________________________________________________
lstm_2 (LSTM)                (None, 11, 6)             1704      
=================================================================
Total params: 52,904
Trainable params: 52,904
Non-trainable params: 0
_________________________________________________________________
../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/checkpoint
No LSTM model loaded.
Train on 8932 samples, validate on 993 samples
Epoch 1/20
8864/8932 [============================>.] - ETA: 0s - loss: 0.0014 - mean_squared_error: 0.0014
Epoch 00001: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 8s 842us/sample - loss: 0.0014 - mean_squared_error: 0.0014 - val_loss: 0.0013 - val_mean_squared_error: 0.0013
Epoch 2/20
8896/8932 [============================>.] - ETA: 0s - loss: 0.0012 - mean_squared_error: 0.0012
Epoch 00002: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 658us/sample - loss: 0.0012 - mean_squared_error: 0.0012 - val_loss: 0.0012 - val_mean_squared_error: 0.0012
Epoch 3/20
8864/8932 [============================>.] - ETA: 0s - loss: 0.0012 - mean_squared_error: 0.0012
Epoch 00003: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 662us/sample - loss: 0.0012 - mean_squared_error: 0.0012 - val_loss: 0.0011 - val_mean_squared_error: 0.0011
Epoch 4/20
8864/8932 [============================>.] - ETA: 0s - loss: 0.0011 - mean_squared_error: 0.0011
Epoch 00004: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 665us/sample - loss: 0.0011 - mean_squared_error: 0.0011 - val_loss: 0.0010 - val_mean_squared_error: 0.0010
Epoch 5/20
8864/8932 [============================>.] - ETA: 0s - loss: 0.0010 - mean_squared_error: 0.0010
Epoch 00005: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 661us/sample - loss: 0.0010 - mean_squared_error: 0.0010 - val_loss: 9.6235e-04 - val_mean_squared_error: 9.6235e-04
Epoch 6/20
8864/8932 [============================>.] - ETA: 0s - loss: 9.5987e-04 - mean_squared_error: 9.5987e-04
Epoch 00006: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 664us/sample - loss: 9.5950e-04 - mean_squared_error: 9.5950e-04 - val_loss: 9.3958e-04 - val_mean_squared_error: 9.3958e-04
Epoch 7/20
8864/8932 [============================>.] - ETA: 0s - loss: 9.3971e-04 - mean_squared_error: 9.3971e-04
Epoch 00007: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 656us/sample - loss: 9.4036e-04 - mean_squared_error: 9.4036e-04 - val_loss: 9.2072e-04 - val_mean_squared_error: 9.2072e-04
Epoch 8/20
8864/8932 [============================>.] - ETA: 0s - loss: 9.2281e-04 - mean_squared_error: 9.2281e-04
Epoch 00008: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 663us/sample - loss: 9.2169e-04 - mean_squared_error: 9.2169e-04 - val_loss: 9.0257e-04 - val_mean_squared_error: 9.0257e-04
Epoch 9/20
8864/8932 [============================>.] - ETA: 0s - loss: 9.0478e-04 - mean_squared_error: 9.0478e-04
Epoch 00009: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 654us/sample - loss: 9.0473e-04 - mean_squared_error: 9.0473e-04 - val_loss: 8.8796e-04 - val_mean_squared_error: 8.8796e-04
Epoch 10/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.9285e-04 - mean_squared_error: 8.9285e-04
Epoch 00010: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 660us/sample - loss: 8.9289e-04 - mean_squared_error: 8.9289e-04 - val_loss: 8.7809e-04 - val_mean_squared_error: 8.7809e-04
Epoch 11/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.8401e-04 - mean_squared_error: 8.8401e-04
Epoch 00011: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 658us/sample - loss: 8.8362e-04 - mean_squared_error: 8.8362e-04 - val_loss: 8.7047e-04 - val_mean_squared_error: 8.7047e-04
Epoch 12/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.7461e-04 - mean_squared_error: 8.7461e-04
Epoch 00012: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 660us/sample - loss: 8.7509e-04 - mean_squared_error: 8.7509e-04 - val_loss: 8.6289e-04 - val_mean_squared_error: 8.6289e-04
Epoch 13/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.6703e-04 - mean_squared_error: 8.6703e-04
Epoch 00013: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 667us/sample - loss: 8.6689e-04 - mean_squared_error: 8.6689e-04 - val_loss: 8.5333e-04 - val_mean_squared_error: 8.5333e-04
Epoch 14/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.5853e-04 - mean_squared_error: 8.5853e-04
Epoch 00014: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 666us/sample - loss: 8.5915e-04 - mean_squared_error: 8.5915e-04 - val_loss: 8.4534e-04 - val_mean_squared_error: 8.4534e-04
Epoch 15/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.5224e-04 - mean_squared_error: 8.5224e-04
Epoch 00015: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 657us/sample - loss: 8.5176e-04 - mean_squared_error: 8.5176e-04 - val_loss: 8.3922e-04 - val_mean_squared_error: 8.3922e-04
Epoch 16/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.4424e-04 - mean_squared_error: 8.4424e-04
Epoch 00016: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 666us/sample - loss: 8.4529e-04 - mean_squared_error: 8.4529e-04 - val_loss: 8.3094e-04 - val_mean_squared_error: 8.3094e-04
Epoch 17/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.3816e-04 - mean_squared_error: 8.3816e-04
Epoch 00017: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 658us/sample - loss: 8.3824e-04 - mean_squared_error: 8.3824e-04 - val_loss: 8.2518e-04 - val_mean_squared_error: 8.2518e-04
Epoch 18/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.3225e-04 - mean_squared_error: 8.3225e-04
Epoch 00018: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 663us/sample - loss: 8.3169e-04 - mean_squared_error: 8.3169e-04 - val_loss: 8.1794e-04 - val_mean_squared_error: 8.1794e-04
Epoch 19/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.2609e-04 - mean_squared_error: 8.2609e-04
Epoch 00019: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 655us/sample - loss: 8.2538e-04 - mean_squared_error: 8.2538e-04 - val_loss: 8.1092e-04 - val_mean_squared_error: 8.1092e-04
Epoch 20/20
8864/8932 [============================>.] - ETA: 0s - loss: 8.1935e-04 - mean_squared_error: 8.1935e-04
Epoch 00020: saving model to ../experiments/local-results/NAB/machine_temp/batch-32/NAB-machine_temp-48-12-6-fixedSigma-0.1/checkpoint/lstm/cp.ckpt
8932/8932 [==============================] - 6s 660us/sample - loss: 8.1929e-04 - mean_squared_error: 8.1929e-04 - val_loss: 8.0707e-04 - val_mean_squared_error: 8.0707e-04
(993, 11, 6)

-none-

-None-

I have tried to reproduce the code

I have
tensorflow 1.15.0
tensorflow-gpu 1.15.2
tensorflow-probability 0.7.0
cudatoolkit 10.0.130
cudnn 7.3.1

but i have this error in "lstm_model.produce_embeddings(config, model_vae, data, sess)"

UnknownError: 2 root error(s) found.
(0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node encoder/conv2d/Conv2D}}]]
(1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
[[{{node encoder/conv2d/Conv2D}}]]
[[encoder/code_mean/BiasAdd/_7]]
0 successful operations.
0 derived errors ignored.

Need Help

Hello , I have read your paper --- Anomaly Detection for Time Series Using VAE-LSTM Hybrid Model . I am curious about the exact structure of your model and I am doing some research on Anomaly Detection for Time Series , so I click the link you leave on your paper . However , I find nothing but a readme in the repository .
If you have no spare time to post your code, could you please tell me the detail of your model's structure.
Best wishes to you.

About the delay in the anomaly detection

During the evaluation I found the detected anomalies tend to lag for a while after the true anomalies.
This phenomenon was also mentioned in your paper: "A potential downside of our method is the delay in the anomaly detection for some cases."
I'm wondering if this is due to the large detect window size, e.g., in your demo the window size is 48*12=576, which means when an anomaly occurs at time t, the prediction error is calculated over time window [t-576, t]. The anomaly will be averaged out by this large time window.

Can you share the code for preprocessing and explain meaning of each index of data?

Can you share the code used to preprocess the npz files in the dataset folder?
And can you explain in detail the meaning of each index of data [below]?

[below]
data = np.load('./machine_temp.npz', mmap_mode='r', allow_pickle=True)
for i, k in enumerate(data.files):
print("i:{}, k:{}".format(i, k))
==========result==========
i:0, k:t
i:1, k:t_unit
i:2, k:readings
i:3, k:idx_anomaly
i:4, k:idx_split
i:5, k:training
i:6, k:test
i:7, k:train_m
i:8, k:train_std
i:9, k:t_train
i:10, k:t_test
i:11, k:idx_anomaly_test

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.