How do I know the training is finished for reformulator_and_selector_training about active-qa HOT 3 CLOSED

1. reformulator_and_selector_training should plateau at ~47.4 F1 score after 1 epoch (or ~5k minibatch updates). Also, you can follow the training progress by lauching tensorboard:
   tensorboard --logdir=$OUT_DIR/tensorboard
   Then navigate your web browser to http://localhost:6006

2. We will provide code to save/load selector checkpoints soon, along with a fully trained selector checkpoint. If you don't want to wait, you can call load() and save() functions in the px/selector/selector_keras.py accordingly.

from active-qa.

@rodrigonogueira4 Did you happen to use 1 epoch for reformulator training too?

from active-qa.

No, the reformulator takes 20-100 epochs to reach 42.4 F1 on the dev set, which is about a week of training on a single P100 GPU.

from active-qa.