halfish / jianzhu Goto Github PK

image-text matching

refer to Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models

# install gensim
pip install gensim

# pip install hyperboard
# refer to https://github.com/WarBean/hyperboard

# install pytorch
# refer to https://github.com/pytorch/pytorch

pip install torchvision

wrap images and sentences in datautil.py

The model was specified in model.py, run

CUDA_VISIBLE_DEVICES=2 python main.py

to train an encoder-decoder model, we will get encoder.pt, then move it to models directory:

>>> mv encoder.pt static/models/

dump static images and sentences for web server retrieval, run

CUDA_VISIBLE_DEVICES=2 python dump_static_data.py

you will get image_static.npy and sentence_static.npy, then move them to models directory,

>>> mv *_static.npy static/models/

python server.py