is this fully_quantized model? about pocketflow HOT 9 CLOSED

• For compressed models generated by UniformQuantTFLearner, all the operations that match TensorFlow's rule for tf.contrib.quantize.experimental_create_training_graph will be quantized. For MobileNet-v1 and MobileNet-v2, all operations will be quantized; for ResNet models, some operation may not be quantized (at least this is true for TensorFlow-1.10.0).
• For TF-Lite models generated by tools/conversion/export_quant_tflite_model.py with the enbl_post_quant option enabled, all operations will be quantized.
• By "chips", which hardware do you refer to exactly? Do you have the corresponding DL inference engine for that hardware, with integer operations supported?

from pocketflow.

em..I am just working on a project, and the other requires a fully_quantized model to run on their 'chips', but i do not know exactly.

from pocketflow.

In short, you need to know which inference engine they are using, and then try to obtain a fully-quantized model (maybe not from PocketFlow) that can be parsed by that inference engine.

from pocketflow.

Will you give the details of how the quantized model weights are saved and how they run mathmatically? So one can implement the model on some specific devices for inference acceleration.

from pocketflow.

@Aspirinkb
For UniformQuantTFLearner, we are using TensorFlow's quantization-aware training API, i.e.:

tf.contrib.quantize.experimental_create_training_graph(
    input_graph=None,
    weight_bits=8,
    activation_bits=8,
    quant_delay=0,
    freeze_bn_delay=None,
    scope=None
)

to modify the graph and insert quantization operations. You may refer to their documentations to see how it works in details.

from pocketflow.

If I prefer to use UniformQuantLearner of PocketFlow's to quantize models rather than TensorFlow's, where can I get the details of the quantized models about their weights, graphs and so on? Do you give some inference implementations about that？If not, I think it is a necessary feature of PocketFlow. What do you think?

from pocketflow.

I also have a question about Uniform Quantization Learner, is this method just quantize weight and activation? So, the data flows int network is still float type， instead ofquantize bits such as 8 or 4 bits?
thanks!

from pocketflow.

@Aspirinkb
UniformQuantLearner is mainly for research purpose (for instance, to see how well RL can determine the bit-width allocation strategy). An efficient inference implementation heavily relies on the hardware platform and requires lots on work. For efficient deployment, we recommend you to use UniformQuantTFLearner, which can be deployed under the TensorFlow Lite framework.

from pocketflow.

@wangxianrui
What are you referring to with "data flows in the network"? Gradients are in 32-bit full-precision. Model weights have both quantized version (for forward pass and backward pass) and unquantized version (for gradient update). Layer activations are quantized.

from pocketflow.