weight behind clip_prefix_captioning_inference.ipynb and your demo about clip_prefix_caption HOT 1 CLOSED

Hi @Cm744,
I can't really be sure what is the issure here
But I will try to suggest several possible solution

1. Can you make sure only_prefix is False? maybe the default value is not defined well in the code?
2. To save computation time, we used early stopping, If I remember we achieved good results after about 7 epochs. So maybe overfit occur for many epochs?
3. I think we used the resnet clip with the mlp
   python train.py --only_prefix --data ./data/coco/oscar_split_RN50x4_train.pkl --out_dir ./coco_train/ --is_rn
4. You can try an earlier version of the code, such as this. Maybe a bug emerged during the changes

Let me know if it was helpful

from clip_prefix_caption.