Language evaluation: segments in test_2 and val_2 about grounded-video-description HOT 4 CLOSED

@sgarbanti This is just to be consistent with existing works (e.g., DEM and MT). Besides, in the DEM paper, the authors mention the high overlapping rate between the two sets of annotations (tIoU of 70.2%), so if two segments overlap by a large percentage, the caption outcome should be close to each other. But you're right, we could choose to avoid the unnecessary performance loss by only considering val_1 and test_1, just like what we did for grounding. Reporting both would be ideal.

from grounded-video-description.

@LuoweiZhou Ok, thank you very much for your answer.
Just to know, is there a particular reason why you didn't generate captions for all segments?

from grounded-video-description.

@sgarbanti We could, but since val_2 and test_2 have no bounding box annotations, we could not conduct any grounding evaluation. This is the main obstacle. In my opinion, like what you mentioned, a better way to evaluate given GT segments is to consider only val_1 and test_1 (or whatever file your prediction is based on) and the what-we-assumed convention to evaluate on both files is kind of ill-posed. Therefore, we'd encourage you to report evaluation on both file as a fair comparison and at the same time on val/test_1 only for a more truthful outcome.

from grounded-video-description.

@LuoweiZhou Ok i understand, since the issue is resolved I am closing it.
Thank you very much.

from grounded-video-description.