Confusion matrix implementation about review_object_detection_metrics HOT 3 CLOSED

We need to reference very well the computation of the confusion matrix. The way it is implemented is referenced in a paper or published somewhere else?

• In Tensorflow, it's available https://github.com/svpino/tf_object_detection_cm
• In Pytorch, it's available in Yolov5 https://github.com/ultralytics/yolov5/blob/develop/utils/metrics.py
• In python https://github.com/kaanakan/object_detection_confusion_matrix

The very first implementation was the one in Tensorflow, then the one in python and finally the one in Pytorch (the 2nd is inspired by the 1st and the 3nd is inspired by the 2nd one). I did not find any papers referring to this metric.

The confidence and iou thresholds in steps 2 and 3 could be defined by the user, right?
--> yes

We have to guarantee that our results are exactly identical to some benchmarking in different datasets. Do you know if tools such as tensorflow or pytorch already implement the confusion matrix for object detection?
--> cf the repo mentionned above

I'll open a PR in the other repo with the following information

from review_object_detection_metrics.

@benoitdes,

I think the confusion matrix feature is well desired and other users can benefit.

But as the current repository aims to support the metrics in a user interface, I believe we can first implement it in the old repository in an independent script in the root directory, like confusion_matrix.py.

We just have to think about some points:

1. We need to reference very well the computation of the confusion matrix. The way it is implemented is referenced in a paper or published somewhere else?

2. The confidence and iou thresholds in steps 2 and 3 could be defined by the user, right?

3. We have to guarantee that our results are exactly identical to some benchmarking in different datasets. Do you know if tools such as tensorflow or pytorch already implement the confusion matrix for object detection?

Please, open a PR in here, so we can move on the development of the confusion matrix. :)

Thank you for your contribution.

from review_object_detection_metrics.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

from review_object_detection_metrics.