Zero-shot inference about concept-based-interpretability-vlm HOT 4 CLOSED

Hello @nhussein,

Thank you for your interest in our work.
Were you able to use the model weights for zero-shot inference?
In any case, I will prepare a python notebook with the necessary steps to perform zero-shot tasks using our pipeline.

from concept-based-interpretability-vlm.

I loaded the weights for the trained layers and I passed the results from zero-shot Clip to the layers. However, I did not get the same results as the one in baseline obtained from embeddings.
Thank you for the prompt response, will check out the notebook when it’s up.

from concept-based-interpretability-vlm.

I loaded the weights for the trained layers and I passed the results from zero-shot Clip to the layers. However, I did not get the same results as the one in baseline obtained from embeddings. Thank you for the prompt response, will check out the notebook when it’s up.

Hi @nhussein,

I've just uploaded the python notebook for zero-shot inference. You can find it here.

In order to reproduce the results of the paper, you should follow the steps described in the README file.

Thank you.

from concept-based-interpretability-vlm.

Thank you!

from concept-based-interpretability-vlm.