drogozhang / led Goto Github PK

Thanks for your contribution and great work!
I have some questions,
How to train model with customed datasets? I am going to train the whole model with Chinese retrieval datasets.
All I need to do :

1. transform my training data to customized formatted data
2. initialize the backbone bert-like model with Chinese bert-like model such as bert-base-chinese
3. train step by step

Am I right? And what the format for the customized training data?

In your paper, the equation (3) and (8) for loss function read,
𝑝 ∈ {𝑝+ } ∩ N, or 𝑝 ∈ {𝑝+ } ∩ N^mix
I don't know why you use intersection instead of union.
Each batch in your loss function consists of positive passages and negative passages.
So I think it confuses me when you use '∩' to represent intersection here.

I saw in your python code "train_splade_retriever.py",

target = torch.arange(ga_emb_query.shape[0], device=ga_emb_query.device, dtype=torch.long) * num_docs

I don't know for sure why you didn't pass the relevance label of passage-query pair explicitly.
It seems that you still utilize In-Batch-Negative sampling methods instead of BM25 negative as in your paper. So I want to make sure it is right or not, and how could I train this model with BM25 negatives.