object detection support about epic-kitchens-55-lib HOT 3 CLOSED

Hi @skrish13,
You are correct. The library's main focus is action recognition at the moment, although it'd be nice to expand it out to support object detection as well. What sort of functionality were you after? One thing I'd like to do is to add metrics to the library, so the object detection metrics would be exposed as part of that.

We use the Tensorflow Object Detection API for metrics and we load the frames into TFRecords for training the baseline model.

@hazeld anything you'd like to add that might be useful? If you've got any ideas on supporting OD in the lib I'd like to hear them

from epic-kitchens-55-lib.

Can you provide any instructions on using the dataset and annotations for training an object detector such as YOLO on your dataset ?
Thanks

from epic-kitchens-55-lib.

The training annotations are accessible through epic_kitchens.meta.training_object_labels() but that is the extent of support provided by this library. The frames are annotated with active object (i.e. the object is participating in the current action) annotations every 1/2 second (every 30 frames). I'd suggest using these annotations in combination with the instructions provided for supporting new datasets by YOLOv3/CenterNet/FasterRCNN/. I sadly don't have any experience with these. Good luck 👍

from epic-kitchens-55-lib.