High-level question: is SAM-HQ suited for fine-tuning on specific tasks? about sam-hq HOT 2 OPEN

Hi Simon, for task-specific finetuning of SAM, you can allow more model parameters to be trainable based on hq-sam's current training setup, such as adding some lora layers into the mask decoder or image encoder, or just unfreeze the whole mask decoder or more blocks in the image encoder. Also, I would suggest to set hq_token_only = True during the inference after successfully finetuning the model.

from sam-hq.

Thanks for your answer !

I also have another question : in my task, each image is endowed with various masks ; think about items like people on a picture. Each picture can contain between 1 and 10 people. So currently, for each image (image.jpg) I have a binary mask (image.png) gathering all the different masks ; would you recommend splitting this mask into many different masks ?

from sam-hq.