Result different from demo_vis_features.ipynb about sd-dino HOT 4 CLOSED

Thanks, I got the output with single background similar to how you had provided in the jupyter notebook. Also, thanks for the explanation.

from sd-dino.

Hi,

I think this is mainly because the instance mask is not generated/loaded correctly, therefore, the background feature map is taken into account when computing the PCA and clustering.

You could refer to this #1 issue on how to generate the correct mask. Please feel free to let me know if you have any further issues.

Best,
Junyi

from sd-dino.

Hello, I added the parameter argument '''decoder= False''' in load_model and the result has improved and the clustering is looking like this

It was written that this mask is only for visualization purposes. So, this means if we need the features for some downstream tasks, we don't need the mask. Right? Also, you had mentioned in issue #1 that we can use SAM to obtain mask. So, the SAM mask will basically replace the get_mask function output?

from sd-dino.

Hi @DevikalyanDas ,

Sorry for the late reply.

As I could see, the background is still not masked (otherwise, it should be single colored). I would suggest using other segmentation methods to generate the mask.

Also, you had mentioned in issue #1 that we can use SAM to obtain mask. So, the SAM mask will basically replace the get_mask function output?

Yes, that's right. You could try to use different ways to get the instance mask.

So, this means if we need the features for some downstream tasks, we don't need the mask. Right?

That's right. The mask is only used for removing the background feature when visualizing the feature maps.

from sd-dino.