ensta-u2is / ldu Goto Github PK

Latent Discriminant deterministic Uncertainty [ECCV2022]

License: GNU General Public License v3.0

Python 98.48% Shell 0.14% MATLAB 1.39%

deep-neural-networks out-of-distribution-detection uncertainty-estimation classification monocular-depth-estimation semantic-segmentation eccv2022

ldu's Introduction

LDU: Latent Discriminant deterministic Uncertainty

PyTorch implementation for Latent Discriminant deterministic Uncertainty (ECCV 2022).
Paper

Abstract

In this work we advance a scalable and effective Deterministic Uncertainty Methods (DUM) that relaxes the Lipschitz constraint typically hindering practicality of such architectures. We learn a discriminant latent space by leveraging a distinction maximization layer over an arbitrarily-sized set of trainable prototypes.

Overview of LDU: the DNN learns a discriminative latent space thanks to learnable prototypes. The DNN backbone computes a feature vector z for an input x and then the DM layer matches it with the prototypes. The computed similarities reflecting the position of z in the learned feature space, are subsequently processed by the classification layer and the uncertainty estimation layer. The dashed arrows point to the loss functions that need to be optimized for training LDU.

For more details, please refer to our paper.

Note

We currently only provide the codes for toy example, classification and monocular depth estimation.
The semantic segmentation part will be released in near future.

Experiment

Toy example

We provide a toy example for illustrating LDU on two-moon dataset.

Monocular depth estimation example

In folder monocular_depth_estimation/, we provide the codes and instructions for LDU applying on monocular depth estimation task. The detailed information is shown on monocular_depth_estimation/README.md.

TODO

Add classification codes

Citation

If you find this work useful for your research, please consider citing our paper:

@article{franchi2022latent,
  title={Latent Discriminant deterministic Uncertainty},
  author={Franchi, Gianni and Yu, Xuanlong and Bursuc, Andrei and Aldea, Emanuel and Dubuisson, Severine and Filliat, David},
  journal={arXiv preprint arXiv:2207.10130},
  year={2022}
}

ldu's People

Contributors

Stargazers

Watchers

Forkers

dijkstra14 jxzhangjhu akola-mbey-denis anhquancao qinghaizheng1992

ldu's Issues

Silog_loss issue

Hi Xuanlong,

in your loss function silog_loss, you take the log of masked depth estimation and gt.
However, after masking, there will be many zeros which will result in -inf, how would you tackle this issue?

Best regards,

Wayne

Clarification - Uncertainty for Classification

Hi there! Hope all is well! Thank you for sharing your code! :) Kindly just checking in, for a classification model - and the one here - I was wanting to find out can one follow the same collab file you shared? I don't see how the paper MCP option works in the code? Sorry if I missed something.

semantic segmentation code

Hello!
Your method is amazing and makes sense. Could you please make the semantic segmentation code public? I would be grateful if it could be made public.

segmentation part

Hi, it is a nice job! I am wondering when the segmentation part could be released. Thanks!

Question about dissimilar_loss

Hi, thanks for your excellent work!
In the dissimilar_loss

loss = -1 * torch.mean(torch.cdist(protos, protos))

What is calculated here is the distance between A and A, which is always 0. Does it seem inconsistent with the paper?

pos_weight for segmentation task

Hello, you are doing a great job. I have some questions and I hope to get your response.

I notice that there is a pos_weight in the uncertainty loss (nn.BCEWithLogitsLoss) but it seems not to be detailed in the paper. In particular, it is 0.01 in the toy example and 5 in the deep estimation task. I want to know how to determine this parameter? and what you set in the segmentation task?

L_unc loss function collapse

Hello, I'm trying to reproduce a semantic segmentation task. The inputs to the L_unc loss function are the confidence score and the result of the task loss function (Cross Entropy with reduction='none'), which have the same shape [B, H, W]. Is that so?
But during the training process, I found that the L_unc loss function value dropped to -3429.994, and the segmentation results also became collapsed. What is the reason for this?
I would be very grateful if I could get your reply.

code for segmentation task

Hi, thanks for sharing your solid work. I am tring to reproduce your method on the segmentation task. Wondering if the training code for segmentation task will be released ? Thanks for your attention.

Nan loss during training

Has anyone ever met Nan lossing during training for bts_main?
Any possible solution for that?

NYU Test data

Hi,

The test file for nyu dataset, has unclear file path, which makes testing unavaliable.
For example, in the file 'nyudepthv2_test_files_with_gt.txt', the first row is: bathroom/rgb_00045.jpg bathroom/sync_depth_00045.png 518.8579;
However, there is no such folder named bathroom, in the sync dataset you provided.
Would you mind giving me a bit hint on that?

Thanks
Wayne