Undetected keypoint confused with the point of (0, 0) about coco-analyze HOT 8 CLOSED

@yw155 - Thanks for pointing this out. Could you please share a bit more details on how to reproduce this bug? Is this happening in the localization errors function? And what version of the COCO dataset are you using (i.e. 2014 or 2017)? I'll look into it right now, but the more info you give me the easier it'll be to spot it.

from coco-analyze.

Hi @matteorr, thanks for your reply. I just found the above outputs from the folder of 'localization_errors/keypoints_breakdown'. And I use COCO 2017. Is it possible that the bug exists after the OKS matching and evaluation considers the point of (0, 0)? Thanks.

from coco-analyze.

I'll look into it.Could you post or send me a link to where I can download your json detections?

from coco-analyze.

Hi @matteorr, the file is here:
coco_val2017-1_4.zip

from coco-analyze.

Hey @yw155, this commit fixes the problem.

The reason the visualizations contained keypoints in (0,0) is that your algorithm predicts keypoints that are not visible (i.e. visibility flag is 0) in location (0,0), while all of the algorithms I previously analyzed always make a prediction for all the keypoints in the skeleton, regardless of the value of the visibility flags.

You now have the option in the localizationErrors file to set a variable USE_VISIBILITY_FOR_PLOTS such that when plotting the detections the value of the visibility flag will be taken in consideration and only keypoints whos visibility is 1 (visible but occluded), or 2 (visible) will be plotted.

NOTE: This bug isn't really a bug, but more of a difference in the outputs of the algorithms, and only affects the plotting but not the evaluation and analysis itself since that already takes into account the visibility. Nonetheless thanks for pointing it out!

from coco-analyze.

Hi @matteorr, I got that. Thanks for your detailed analysis. The commit is great.

from coco-analyze.

Just hit this, it might be good to put a comment about this on the readme?

from coco-analyze.

@WillBrennan - Great suggestion! Thanks for your contributions to the repo, small details can make a big difference.

from coco-analyze.