How to load bert embeddings about gnrs HOT 2 CLOSED

You should prepare bert token embeddings yourself. One approach is to use transformers package by:

from transformers import BertModel

bert = BertModel.from_pretrained('bert-base-uncased')
embeds = bert.embeddings.word_embeddings.weight.detach().numpy()

import numpy as np

np.save(embeds, 'bert-token.npy')

The above code is just a illustrative draft and has not been tested.

In this case, the embed configuration should be:

name: bert-token  # can be any name
embeddings:
  -
    vocab_name: english
    vocab_type: numpy
    path: /path/to/bert-token.npy
    frozen: true  # not trainable

from gnrs.

Besides, the configs look a little bit confusing to me. If i try to load bert embedding and not use image features, can i use the following config?
mind.yaml-->dcn/din/bst/pnn.yaml-->tt.yaml-->bert-token.yaml

Sure! Recommend using tt-ctr.yaml for CTR models such as DCN and PNN.

from gnrs.