About control signal prediction about adapt HOT 3 OPEN

We thank a lot for these constructive suggestions and comments.

1. This is a good suggestion. But in our implement, we use a transformer encoder to directly predict control signals from input video images​. A transformer decoder may work better.
2. This is a good question, and we find this is a bug for adapt, where we ignore the fps of the video after sampling. Thus a good solution is to add an temporal embedding that utilize fps as input. We would fix this bug soon.​

from adapt.

We thank a lot for these constructive suggestions and comments.

1. This is a good suggestion. But in our implement, we use a transformer encoder to directly predict control signals from input video images​. A transformer decoder may work better.
2. This is a good question, and we find this is a bug for adapt, where we ignore the fps of the video after sampling. Thus a good solution is to add an temporal embedding that utilize fps as input. We would fix this bug soon.​

Hello, the uniform sampling method used in paper, does fps calculation use video duration divided by 2，4，8，16 or 32

from adapt.

Yes, this is right and you can also use video duration as input. But this bdd-x is too small and it is easy to overfit. Thus I suggest doing cropping on the duration of the video to make the video more consistent.

from adapt.