Question about "init_checkpoint" and "output_dir" checkpint about bert-gpu HOT 9 CLOSED

感谢~~，it works。
按照您的意思，总的来说应该是这样的：

1. Init_checkpoint是为了确定哪些参数是可以restore from the checkpoint的，也就是后面log 中有 INIT_FROM_CKPT的

2. output_dir 里的模型位置可以用来填充Init_checkpoint参数，如果两个参数都有的话，应该先从checkpoint确定哪些参数可以restore from the checkpoint, 应该是后面restore的时候，带INIT_FROM_CKPT的参数会从output_dir中模型中restore。

不知理解是否正确。

from bert-gpu.

It is a good question, you should be careful for them. The only thing you keep in mind is when you are working based on this project, you should make sure that the model is loaded right.

In fact I do not know the answer exactly. Something may not right but helpful:
https://blog.csdn.net/guotong1988/article/details/100539565

from bert-gpu.

In summary,
I guess init_checkpoint is for the training of beginning.
output_dir is for the predicting.
You should delete output_dir for training and delete the init_checkpoint for predicting, in order to make everything exact right.

from bert-gpu.

Thx~, I see that, It's helpful for me to understand the function.
But is there any right way to restore a interupted model and continue to train (modify num_train_steps = 300000 to num_train_steps= 600000)?
When I fill out both of the parameter with init_checkpoing = 'google_bert_model.ckpt' and output_dir = 'my_finetuing_bertmodel.ckpt', I found that the loss of the model is not cut off, but continuing, but higher than before step by step. I don't know is this right. Do you have any suggestion?

from bert-gpu.

参考 https://blog.csdn.net/guotong1988/article/details/88711524

from bert-gpu.

好像要加上-300000之类的

from bert-gpu.

只能说试试，我记得我当时也遇到过这个问题，没深究。

from bert-gpu.

不知道，以实践结果为准。
it works？什么works？

from bert-gpu.

好吧，并不work。。。。Thx~

from bert-gpu.