liuyudut / rrf-net Goto Github PK

"Learning a Recurrent Residual Fusion Network for Multimodal Matching".

please cite this paper, if the codes have been used for your research.

• Please download the Flickr30K and MS-COCO datasets from their websites.
• The data is stored in LMDB, like "image feature + text feature + image ID".
• Each image has five sentence-level descriptions, and so five descriptions corresponds to the same image ID.
• It is necessary to randomly shuffle the data when converting them into LMDB.

• The training prototxt files are in ./models.
• We do not test the model during the training stage.

• extract_features.py: extract the L2-norm image and text embedding featutres.
• computePR.m: evaluate the performance of image-to-text and text-to-image retrieval.

"Learning Deep Structure-Preserving Image-Text Embeddings", CVPR 2016.

"Fisher vectors derived from hybrid gaussian-laplacian mixture models for image annotation", CVPR 2015.

This code is based on Caffe. Thanks to the contributors of Caffe.