What's the training parameters for MobileNetV2 8 Bit Fixed-point? about pocketflow HOT 12 CLOSED

In order to be able to export the compressed model to TF-Lite, please use UniformQuantTFLearner, which is based on TensorFlow's quantization-aware training APIs, instead of UniformQuantLearner.

The detailed command can be:

# local mode
./scripts/run_local.sh nets/mobilenet_at_ilsvrc12_run.py \
  --learner uniform-tf \
  --mobilenet_version 2 \
  --nb_epochs_rat 0.1

# docker mode + 8 GPUs
./scripts/run_docker.sh nets/mobilenet_at_ilsvrc12_run.py -n=8 \
  --learner uniform-tf \
  --mobilenet_version 2 \
  --nb_epochs_rat 0.1

Detailed explaination:

• --learner uniform-tf: use UniformQuantTFLearner for model compression
• --mobilenet_version 2: use MobileNet-v2 (default is MobileNet-v1)
• --nb_epochs_rat 0.1: use 10% training epochs as used in the standard full-precision training (usually 10% is enough for uniform quantization)

from pocketflow.

@jiaxiang-wu Thanks for the quick response. If chosen the run_local.sh, is there any other hyper-parameter changes needed to reproduce the 72% accuracy, for example initial learning rate?

Also, saw there are a typo in the https://pocketflow.github.io/tutorial/, "find 4,096 training files" should be "find 1,024 training files" based on this link: https://github.com/tensorflow/models/tree/master/research/inception#getting-started.

from pocketflow.

The above command should be enough. No more hyper-parameter needed. Thanks for pointing out the typo.

from pocketflow.

TODO: change the number of training files in the tutorial from 4096 to 1024.

from pocketflow.

@jiaxiang-wu Thanks, here is the results after training more than 12 hours using local mode. The Top1 accuracy is very low, and the loss is still very high, also why the optimal weight quantization is shown as 4? Saw the published accuracy number for the MobileNetV2 is 8 bit (MobileNet-v2 | 71.84% | 90.60% | 72.26% (+0.42%) | 90.77% (+0.17%)), right?

INFO:tensorflow:loss: 10.742256164550781
INFO:tensorflow:accuracy: 0.01297999918460846
INFO:tensorflow:Optimal Weight Quantization:[4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4, 4]

And also add the intermediate training logs:

.0503 | acc_top1 = 0.0000 | acc_top5 = 0.0156 | speed = 172.22 pics / sec
INFO:tensorflow:iter #142100: lr = 4.500000e-07 | model_loss = 7.2648 | loss = 7.2648 | acc_top1 = 0.0000 | acc_top5 = 0.0312 | speed = 170.89 pics / sec
INFO:tensorflow:iter #142200: lr = 4.500000e-07 | model_loss = 7.1522 | loss = 7.1522 | acc_top1 = 0.0000 | acc_top5 = 0.0312 | speed = 172.06 pics / sec
INFO:tensorflow:iter #142300: lr = 4.500000e-07 | model_loss = 7.1517 | loss = 7.1517 | acc_top1 = 0.0000 | acc_top5 = 0.0625 | speed = 172.49 pics / sec
INFO:tensorflow:iter #142400: lr = 4.500000e-07 | model_loss = 6.9467 | loss = 6.9467 | acc_top1 = 0.0312 | acc_top5 = 0.0469 | speed = 174.34 pics / sec
INFO:tensorflow:iter #142500: lr = 4.500000e-07 | model_loss = 7.1331 | loss = 7.1331 | acc_top1 = 0.0000 | acc_top5 = 0.0312 | speed = 173.76 pics / sec

BTW, I followed the exact local mode command as you mentioned for the above training results.

from pocketflow.

@jiaxiang-wu I knew the root cause why the weight quantization is 4 bits now, the typed cmd missed the "-tf" due to the "-" between uniform and tf. It's working properly now. Thanks.

from pocketflow.

@terrydlca Good to know it works.

from pocketflow.

@jiaxiang-wu Another question is there are errors when convert the trained mobilenet_v2 model to tflite format. Do you have an example cmd for mobilenet_v2 tflite conversion?

python tools/conversion/export_quant_tflite_model.py --model_dir ./models_uqtf

Traceback (most recent call last):
File "tools/conversion/export_quant_tflite_model.py", line 271, in
tf.app.run()
File "tools/conversion/export_quant_tflite_model.py", line 249, in main
input_name, input_shape = get_input_name_n_shape(file_path_meta)
File "tools/conversion/export_quant_tflite_model.py", line 63, in get_input_name_n_shape
net_input = tf.get_collection(FLAGS.input_coll)[0]
IndexError: list index out of range

from pocketflow.

@terrydlca
You should specify the save path for the evaluation graph, instead of the training graph, of the compressed model, which can be exported to the TF-Lite model.

$ python tools/conversion/export_quant_tflite_model.py --model_dir ./models_uqtf_eval

from pocketflow.

@jiaxiang-wu Thanks, it works now.

from pocketflow.

Resolved in the pull request #66

from pocketflow.

hi jiaxiang-wu
where will be the compressed model saved??i.e weigts and cfg file for testing model

from pocketflow.