ts-kim / revin Goto Github PK

What if the x_in and x_out have different numbers of features?

For example, we use multivariable to predict a single variable. They don't have the same number of channels.

Thanks

I added RevIN in the Informer according to your method, but the effect is not obvious, you can see if I add it correctly

   def forward(self, x_enc, x_mark_enc, x_dec, x_mark_dec, enc_self_mask=None, dec_self_mask=None, dec_enc_mask=None):
            x_enc = self.revin_layer(x=x_enc,mode='norm')
            x_dec = self.revin_layer(x=x_dec,mode='norm')
            enc_out = self.enc_embedding(x_enc, x_mark_enc)
            enc_out, attns = self.encoder(enc_out, attn_mask=enc_self_mask)
            dec_out = self.dec_embedding(x_dec, x_mark_dec)
            dec_out = self.decoder(dec_out, enc_out, x_mask=dec_self_mask, cross_mask=dec_enc_mask)
            dec_out = self.projection(dec_out)
            dec_out = self.revin_layer(x=dec_out,mode='denorm')
            
    
            if self.output_attention:
                return dec_out[:,-self.pred_len:,:], attns
            else:
                return dec_out[:,-self.pred_len:,:] # [B, L, D]

Hi,

I appreciate your paper and code.

What I am confused about is why there is no comparison of rev layer norm (similar to RevBn, RevIN) in Table 10 of the appendix？

thank you!

Hi,

I appreciate your paper and code.

Could you confirm if you standardized the datasets (applied z-scores) before using RevIN?

Thank you,
Huy

Will you release the code for training N-BEATS model with RevIN?

I would love to add this to the darts forecasting project . Can you add a compatible license (MIT or apache 2.0) so I can in included?

Hello, thank you very much for the code you provided, but I would like to ask, why sometimes adding RevIN in SCINet performs better when the length of the input sequence is different from the length to be predicted for my own data set, but in the same data Under the set, change the length of the input sequence, and the performance of adding RevIN is not good.

Thank you for such great work. I'm working on time series anomaly detection, and your work has inspired me a lot.
But I have a question about the denormalization in DAIN. You said

However, a denormalization step cannot be added to DAIN since it has the Hadamard multiplication operation in the last step, which is not reversible when the input and prediction sequence lengths are different.

in A.7. But I guess it uses the same gating coefficient $\gamma^{(i)}$ in all timesteps, so it seems to be reversible too, with a corresponding Hadamard (entrywise) division operator.

Dear author, I want to ask you a question. Your method uses Z-core normalization in the data processing stage. This normalization method and the method you proposed are used at the same time. What is the difference between your method and this normalization method? Is the difference between two more learnable affine parameters? Are these two parameter values equal to the mean and the variance mentioned in your paper?

Thank you for making such a great contribution to the normalization！
I have some questions about Figure 3 in the paper. The proposed method RevIn is to normalize the samples of a batch in the network, Does Figure 3(Effect of RevIN on distribution discrepancy between training and test data) depict the data distribution of the entire training set and test set? Here's how it's done.

Please tell me, did I get it right here?
..for some reason, the results even got worse.

import torch.nn as nn
import torch.optim as optim

class MultipleRegression(nn.Module):
def init(self, num_features):
super(MultipleRegression, self).init()

    self.layer_1 = nn.Linear(num_features, 16)
    self.layer_2 = nn.Linear(16, 32)
    self.layer_3 = nn.Linear(32, num_features)
    self.layer_out = nn.Linear(num_features, 1)
    
    self.relu = nn.ELU()  
    self.revin_layer = RevIN(num_features)   # <<<-----

def forward(self, inputs):
    x_in = self.revin_layer(inputs, 'norm')
    
    x = self.relu(self.layer_1(x_in))
    x = self.relu(self.layer_2(x))
    x = self.relu(self.layer_3(x))
    
    x = self.revin_layer(x, 'denorm')
    x_out = self.layer_out(x)

    return x_out

I have difficulty implementing RevIN on Informer.
I'm confused about what to do with the Decoder input of Informer, so I would like to refer to the official code.

Thank you for the code.

I did a quick check with revin to see if it is causal along the sequence length axis, where by "causal" I mean that only the past and current tokens are used for transforming the token (or in other words that the future tokens cannot impact past). My results show that RevIN is not causal.

Can you confirm that this is correct? Thanks!