Tensor transform triggers dequantization about compute-engine HOT 6 CLOSED

Thanks! Although I did not get to debug the benchmark tool with the command you provided, I succeeded by building it with:

bazel build //larq_compute_engine/tflite/benchmark:lce_benchmark_model --compilation_mode=dbg -s

Then I was able to properly read the stacktrace as you described and found the problem. I can now benchmark my model! Thanks again for the effort, I will close the issue now.

from compute-engine.

Thats a brilliant idea! That I did not think of this myself. Thanks, I guess for now this is the most straightforward method to get my model in the right form for the converter. I am able to produce a TFLite model now, although the benchmark seems to freeze for some reason for my model only. So far it seems that all operators in the TFLite model are supported, so not sure where this problem is originating. I will spend a bit of time investigating this and open a new issue if needed.

from compute-engine.

I am currently suspecting that it has to do with the pattern matching in compute_engine/mlir/transforms

That is correct. The way it works is as follows: first the ste_sign is transformed into a pair LceQuantize > LceDequantize. Then in a separate step, LceDequantize > Pad > Conv2D is converted into an LceBconv2d. The end result is LceQuantize > LceBconv2d.

I think that if you move the resize and transpose before the ste_sign then it should work. Is that possible in your situation?

from compute-engine.

Thanks for your fast reply. I have found the pattern matching in prepare_patterns_common.td
As said I don't think I can manage to move these steps to the front in my tensorflow code. Therefore, my idea was to see if I can perhaps do some custom pattern matching so that LceDequantize > Resize > Transpose > Pad > Conv2D gets converted into LceQuantize > LceBconv2d. Do you think such a thing is possible?

from compute-engine.

The LceBconv2d op only accepts NHWC input, so if we would add extra pattern to the converter that "absorbs" the Transpose into the LceBconv2d then it would not run at runtime. It might work if we make a pattern that moves the Transpose in front of the LceDequantize. Then the final result would be LceQuantize > Reshape > Transpose > LceBconv2d.

That would be quite a bit of work, so its probably worth it to first try and fix it on the Python side by changing the model.

I don't know the details of your model, but perhaps it is possible to first train your model as you have it now (which does not convert properly). Then once it is trained you no longer need gradients so you could create an "inference-only version" of the same model where you move the transpose in front of the ste_sign but load the weights of the original model. Is that something that could work? If not, maybe you can share a small snippet of the model code.

from compute-engine.

If you're able to share the .tflite file (with randomized weights is fine) I'd be happy to run lce_benchmark in a debugger to see why it freezes.

EDIT: You can do this as follows if you have bazel installed:
First build a 'debug' version of the benchmark binary using bazel test

bazel test //larq_compute_engine/tflite/benchmark:lce_benchmark_model

Then run it in GDB like this:

gdb --args bazel-bin/larq_compute_engine/tflite/benchmark/lce_benchmark_model --graph=/path/to/your/model.tflite

In gdb simply enter run<enter> and when it crashes, enter backtrace<enter>. That information should help us debug the problem.

from compute-engine.