Hi,

I like your jupyternotebook implementaition.
It is very clear.
However, I see the prediction performance of gmm is not better than mlp in the out-of-ditribution area.
Did I miss anything?

Best

Hi,
I was looking for an implementation of Deep Ensambles and I've found your Notebook. Have you found out why you couldn't reproduce some results of the Deep Ensembles paper in the end?
Thanks!

Hi, I am PhD student in South Korea, making use of your deep ensemble python code actively.

While using your Python code, I think I can answer your questions in the end of the ipynb file.

Q

1. Is something wrong with the training procedure in terms of batchsize, epochs? In the paper, it was mentioned, that a batchsize of 100 was used, but the toy example has only 20 samples. Training on these 20 samples for 40 epochs leads to underfitting for the NLL loss.
   --> In the Algorithm 1 in the original paper, it is stated that the "single nm for clarity, minibatch in practice".
   Therefore, in the regression on toy datasets, I think that minibatch size of 1 should be applied.
   Then, I think the underfitting issue will be solved.

2. Is something wrong with the NLL loss function?
   --> I think your "def NLLloss(y, mean, var):" is correct.

3. Is something wrong with the standard initialization in PyTorch? From Figure 1 (right) in the paper, I argue that each network of the ensemble has a more varying output outside the interval [-4, 4], than it is the case in my reproduction. I can't reproduce the predictive uncertainty as seen in the figure above.
   --> In Figure 1 of the original paper, it is stated that the confidence interval was drawn with 3std.
   However, in your code, the plot is set to draw a confidence interval with 1std.