sigma_weight about pytorch-bayesiancnn HOT 1 CLOSED

Hi @Yuzhi-Yang,

We do a forward pass and we sample from the activations using local reparameterization trick and starting with a normal distribution. This is equivalent to learning the MAP of the variational posterior
probability distribution qθ(w|D). Then we do another conv forward pass and then we sample again. Now the variance in the sampled value with the previous MAP is noted and it acts as the variance between the outputs. We can do technically as many passes as we want and capture the weight to get a better variance but that would be very expensive. So, we limit it. Now the means can be averaged and the variance is what we learn and we keep these mean and variance going forward.

from pytorch-bayesiancnn.