fusion_rules not match about nn-meter HOT 1 CLOSED

Hi, thanks for raising the issue.

First of all, the fusion rules in fusion_rules.json indicates whether two ops will be fused in the corresponding device. When there comes a network for latency prediction, nn-meter will do: 1. parse the network to get the network structure, and convert the network architecture to nn-meter ir graph; 2. detect all kernels based on fusion_rules.json of the predictor, for example, if "bn_relu" and "conv_bn" both obey the fusion ("BF_bn_relu": {"obey": true}, "BF_conv_bn": {"obey": true}), then all "conv + bn + relu" in the network will be fused to a kernel "conv-bn-relu"; 3. go through the network after kernel detection, and predict kernel-wise latency based on the kernel configs.

In your mentioned fusion rules, "CBC" and "RBC" are legacy test cases, so that there is no need to test these two cases when building your own predictor. Furthermore, nn-Meter building tools also removed test cases of fusing three ops such as "conv_bn_relu" and "dwconv_bn_relu", since the kernel "conv_bn_relu" will be detected by "conv_bn" and "bn_relu".

As for your second question, the detected kernel is based on fusion rules but not equal to fusion rules. As I mentioned before, a detected kernel of "conv_bn_relu" should based on two fusion rules, "conv_bn" and "bn_relu". In the released CPU predictor "cortexA76cpu_tflite21", we have tested all fusion rules and summarized all detected kernels in this page.

The third question is about the default predictorbuild_config.yaml. Here we only listed some common kernel for prediction. However, the kernel relies on both the inference framework and the hardware, as both of them impact the fusion rules. We suggest detect fusion rules for your own backend, and then determine all kernels by fusion rule detection results.

The last question about whether to merge some kernels into one predictor, my suggestion is to take a look at the profiling latency. According to our experience, the latency of concat/reshape/add/relu are relatively small when compared with conv/dwconv/fc op (which we call it a dominant op). If you found two kernels with the same dominant op has similar latency results, you could consider to merge the two kernels in one predictor.

from nn-meter.