Resnet20 cifar10 compress experiment cant get a small model about pocketflow HOT 12 CLOSED

After channel pruning, the resulting checkpoint files still save weight tensors in their original sizes, including those all-zero channels. Therefore, the checkpoint files will not be smaller.
We have provided a model conversion script, tools/conversion/export_pb_tflite_models.py, to generate *.pb and *.tflite models that are smaller after channel pruning. See the tutorial documentation for detailed usage.

from pocketflow.

@xiaomr Or you can delete the channels whose elements are all zeros by yourself.

from pocketflow.

Thanks for the explanation, It works! with the help of tools/conversion/export_pb_tflite_models.py, I
got the smaller compressed pb。Here comes a few another question, Firstly, it seems this tool only support the fullprecison mode and dis_chn_pruned mode，because both of them have the collection of images final and logits final which are necessary for this tool, when I use this tool for channel pruning mode, it will throw out error. Second, In my experiment the inference time of fullprecision mode and dis_chn_pruned mode is the same , in details, it seems during convertion it will add many 1*1 convolution for channel selection , which means pruning doesn't get actual speedup in GPU?

from pocketflow.

@jiaxiang-wu @psyyz10

from pocketflow.

1. We are investigating the model conversion issue for channel pruning module.
2. After channel pruning, 1x1 convolution will be added for channel selection and this will bring speed-up for CPU-based inference with TF-Lite, according to our evaluation results. The speed-up on GPU may be negligible.

from pocketflow.

Bug: cannot convert the compressed model from channel pruning module to *.pb & *.tflite models.

from pocketflow.

@jiaxiang-wu , Tank you ! Another observation makes me confused is that in full precision mode, model_transformed.pb is much faster than model_original.pb, about 5X speed up in GPU , the channels are not be pruned, so why the graph edition (seems only contain dropout related edition) can accelerate so much?

from pocketflow.

@xiaomr Can you list the detailed time comparison, and describe how these two models are obtained?

from pocketflow.

@jiaxiang-wu The details are as following:
I set the mode of full precision to train resnet20 cifar10 and I get checkpoint file of
model.ckpt.data-00000-of-00001, model.ckpt.index and model.ckpt.meta in ./model_eval after training, then I use tools/conversion/export_pb_tflite_models.py to convert model, I add inference time test code in test_pb_model function of the script, test the average time of sess.run of several samples except for the first 10 warm up, the tools will generate two pbs files in the directory, which are called model_original.pb and model_transformed.pb, time is 122ms and 20 ms respectly in my GPU , I am confused about the acceleration 。

from pocketflow.

@xiaomr Could you please upload all the files under the models_eval directory, so that we can re-produce your issue?

from pocketflow.

models_eval.zip
@jiaxiang-wu @psyyz10

from pocketflow.

@xiaomr full precision learner will not do any compression, it is just a learner to train a model. Please use other learners to compress the model.

from pocketflow.