Wikipedia Alias' and Bigger Base Model for Training about linktransformer HOT 3 CLOSED

Thanks so much, really appreciate the detailed response. I have been playing with FAISS a lot, inspired by your great package.

from linktransformer.

Thanks, Peter! It appears that we never got notified of this issue via github notifications.

1. We use wikidata (https://www.wikidata.org/wiki/Q487907) to train these models. We are currently working on an updated draft of the paper and will make the data available in a couple of weeks if not before.
2. You can absolutely train a larger model on these! We already have code to do so in the repo (under benchmarking/), once the data is made available, perhaps you can train it with any backbone. We chose smaller models only to demonstrate trainability on free colab instances
3. While we haven't formally tested it, but there are references in the literature talking about problems BERT-based models face with short text, like names. You can absolutely use Open AI embeddings and see what you get! We have examples using OAI embeddings in the notebook. Re: the prefix, It is definitely worth trying out. I'd probably make it "The name of the company is XXXX" instead of the colon format. If the motivation here is to add more language, it would be better achieved with a fuller sentence. Open AI has released two new embedding models as well, we will update their performance on our test split once we update the paper. We are aiming for a newer version of everything by late feb-early March.

Sorry for the delay! Feel free to ask anything else.

from linktransformer.

Hi @lamberpj , a new and vastly improved company model is now up. In fact, all of our models have been updated. We are targeting the completion of a new version of the paper + release of the data within the next week or so.

Closing this for now.

from linktransformer.