Question about the way to use new data for incremental training about amazon-a2i-sample-jupyter-notebooks HOT 1 OPEN

Hello @tkashi ,
Thanks for your feedback. Sorry for a delayed response.

Our object detection algorithm does not require the original (first training) data in the incremental training. You may include the full, a subset or none of original training data into the incremental training. What may have more impact on the performance is the choice of hyperparameter, most critically the learning rate. As the previous model has reached a more stable state and the learning rate most likely has reduced from what you used at the beginning, you would need a smaller starting learning rate to begin the incremental training. If the learning rate is not appropriate, your might again jump out of the optimum in the optimization. I would suggest to perform a hyperparameter tuning to find new best hyperparameters. Additionally, I suggest to mix some if not all old validation data with A2I labeled new data to make sure the model does not deviate from your first model towards new data.

Thanks,
Michael Hsieh

from amazon-a2i-sample-jupyter-notebooks.