download.sh about ns-vqa HOT 5 OPEN

Yea I noticed this as well. I had most of the data offline (other than the huge CLEVR_v1.0 folder which you can obtain here).

Here is a link to the data that I had available locally:

from ns-vqa.

Thank you!

from ns-vqa.

@atharvas Hi, can you reproduce the results when retraining the model?

from ns-vqa.

I didn't retrain the model. However, the pretrained model is pretty good as it stands if used with the modules in reason/executor/clevr-executors.py . I noticed that their model is amazing at picking up shape, color, material, size but its not that great at picking up {x, y, z} coordinates. Here are some observations:

Has the following (formatted) output (gt = ground truth, res = model result):

cy | me  | blu : res = < 0.06,  2.04,  0.71> | gt = < 3.00, -3.00,  0.70>
cy | me  | red : res = < 1.32,  2.75,  0.92> | gt = < 0.00, -3.00,  0.70>
cy | me  | yel : res = < 1.09,  1.51,  0.97> | gt = <-3.00, -3.00,  0.70>
cy | ru  | blu : res = < 1.34,  1.01,  0.71> | gt = < 3.00,  0.00,  0.70>
cy | ru  | bro : res = < 1.13, -1.36,  0.90> | gt = <-3.00,  3.00,  0.70>
cy | ru  | cya : res = < 2.08,  0.21,  0.68> | gt = < 3.00,  3.00,  0.70>
cy | ru  | pur : res = < 2.40, -0.13,  0.83> | gt = < 0.00,  3.00,  0.70>
cy | ru  | red : res = < 1.76,  1.29,  0.96> | gt = < 0.00,  0.00,  0.70>
cy | ru  | yel : res = < 1.77,  0.06,  1.07> | gt = <-3.00,  0.00,  0.70>

Plotting the results on a scatter plot gives the following:

Additionally, the model does not pick up on z value variations (Objects above the plane, or above another object have a nonsensical z value).

TL;DR: Nope. didn't retrain the model.

from ns-vqa.

@atharvas Thanks a lot!

from ns-vqa.