The error when training the graph_retriever in the HotpotQA about learning_to_retrieve_reasoning_paths HOT 5 CLOSED

Hi, thank you for posting the issue! :)
Would you tell us which command did you run when you face this issue? We'd like to reproduce this error in our environment to fix it, as we haven't seen in our environment.
The error might come from this assertion error.

from learning_to_retrieve_reasoning_paths.

Hi, thank you for posting the issue! :)
Would you tell us which command did you run when you face this issue? We'd like to reproduce this error in our environment to fix it, as we haven't seen in our environment.
The error might come from this assertion error.

Thanks for your reply, and the command I used is

python run_graph_retriever.py
--task hotpot_open
--bert_model bert-base-uncased --do_lower_case
--train_file_path ./data/hotpotqa_new_selector_train_data_db_2017_10_12_fix/db=wiki_hotpotqa.db_hotpotqa_new_test_tfidf_k=50.pruning_l=100_tag_me=True.prune_after_agg=False.prune_in_article=False_use_link=True_start=
--output_dir ./train_model/retriever/
--max_para_num 50
--tfidf_limit 40
--neg_chunk 8 --train_batch_size 4 --gradient_accumulation_steps 1
--learning_rate 3e-5 --num_train_epochs 3
--use_redundant
--max_select_num 4 \

I want to train the graph-based recurrent retriever model in the HotpotQA train dataset.

from learning_to_retrieve_reasoning_paths.

Sorry for my late response. It seems that the path to train data may not be correct ./data/hotpotqa_new_selector_train_data_db_2017_10_12_fix/db=wiki_hotpotqa.db_hotpotqa_new_test_tfidf_k=50.pruning_l=100_tag_me=True.prune_after_agg=False.prune_in_article=False_use_link=True_start= but I'm not sure if it's just copying error. I have tried to run training on HotpotQA using our data and codes and I couldn't reproduce the error...

Although I'm not sure if it's the case, I got similar kinds of errors when I was running our model on our own internal data. At that time, I have mistakenly fed inputs with empty context as a TF-IDF retriever failed to return any paragraphs due to some special characters.

from learning_to_retrieve_reasoning_paths.

Hi, I encounter the same error. However, it works after I change --tfidf_limit 40 to --tfidf_limit 60. I wonder why this happens as --tfidf_limit seems only to control the number of negative samples?

------------------------------Update on Dec 1st------------------------------------
I solve the error by updating my python from 3.5 to 3.7. I guess the error is due to how py 3.5 handle the dictionary at here. Dictionaries in py35 are not ordered, so new_context may not contain the ground truth as we expect.

from learning_to_retrieve_reasoning_paths.

Thanks, @sysu-zjw for the update! We've tested the code mostly with python 3.6, so the issue you pointed out might have caused the problem. @Frankszc, I close this issue now but please feel free to re-open.

from learning_to_retrieve_reasoning_paths.