Differences between provided dataset and repo code about dreyeve HOT 3 CLOSED

Hi @Amakri1020 and thank you for interest.

Yes, in order to retrain the network, you should unroll all the .avi sequences (video_garmin.avi and video_saliency.avi) into frames. The code assumes the following structure:

<data_folder>/sequence:02d/frames/<frame_index:06d>.jpg --> for input frames
<data_folder>/sequence:02d/saliency_fix/<frame_index:06d>.png --> for fixation maps

As mentioned in this issue, you should comment out lines of code trying to load annotations from the saliency subfolder (saliency_fix is the one).

Let me know if this helps,
Best,
D

from dreyeve.

Thanks for the quick response, that is helpful!

I am also curious if there were instances where the driver looked at something out of the FoV of the camera and if so, how did you deal with these cases?

from dreyeve.

That's an interesting question :)

It is likely that during the recording a driver took quick peeks outside the FoV (e.g, looking at side mirrors).
Anyway, the effect of rapid shifts in attention is ameliorated by the fixation map construction procedure. Indeed, as mentioned in the journal paper, such a procedure involves a temporal aggregation of fixation points to build a single fixation maps.

Short answer: we don't deal with such cases.
I don't think these situations are encoded in fixation maps in the first place. You could still get them by looking at the ETG videos and the raw fixation recordings. But I'm not sure.

D

from dreyeve.