Unsupported data type in custom op handler about edgetpu-rs HOT 1 OPEN

Hello Azw413,

I fixed this very error a month ago on my Pi 4B running a USB coral edgetpu accelerator. This is probably not the only error you will get. This error is due to differing versions of tensorflow in the libraries you are using.

libedgetpu on your device uses a newer version of tensorflow, while a dependent of this library ionosnetworks/tflite-rs--forked off the official tflite-rs--still uses tensorflow v1.14 while the original repository now uses tensorflow v2.4.1. Simply put this library requires you to use tensorflow v1.14 while the libedgetpu libraries were compiled for another, newer, version of tensorflow.

why this is a problem:
see issue: google-coral/edgetpu#214

How to fix this

You will need to downgrade tensorflow for the libedgetpu libraries: (libedgetpu-dev and either libedgetpu-std or lbedgetpu-max) and relavent headers (edgetpu.h and edgetpu_c.h).

From your file structure(User/..) I believe you are using Unix. The following process worked for my armv7 system. I hope it is still relevant.

On Linux this can be resolved by using a build of the library or manually building it yourself.

Both have worked for me but for replicability (and my own sanity) I wrote a dependencies sh to build and hack all the dependancies I was using on a new system and for gh actions. I personally used these prebuilt armv7a shared library files and headers for libedgetpu.

Pi: If you want to build it yourself you do need another device with bazel installed. While bazel can be installed on an arm device. It too needs to cross-compiled for armv7a. issue: google-coral/edgetpu#12

Mac: While building from source did not make much sense for my use case, it might for yours. See https://github.com/google-coral/edgetpu#macos

After building, the install script did not work entirely correctly, so I had to manually move the shared library files and headers to their final destinations:
The snippet from littleTitan/tiny-object-detection/blob/main/dependencies.sh

sudo rm /usr/lib/$CROSSTC/libedgetpu.so
sudo rm /usr/lib/$CROSSTC/libedgetpu.so.1
sudo rm /usr/lib/$CROSSTC/libedgetpu.so.1.0
git clone https://github.com/google-coral/edgetpu
cd edgetpu
sudo cp libedgetpu/edgetpu.h /usr/include/edgetpu.h
sudo cp libedgetpu/edgetpu_c.h /usr/include/edgetpu_c.h
sudo cp libedgetpu/direct/armv7a/libedgetpu.so.* /usr/lib/$CROSSTC
cd /usr/lib/$CROSSTC
sudo ln libedgetpu.so.1.0 libedgetpu.so

Note: I have not yet taken the time to clean this patch up. The gh actions equivalent is very much a hacky solution. It actually produces this exact runtime error when it is run on the Ubuntu x86-64 system github provides or, initially, on the pi I was working on.

After this you may get an error concerning the version of the edgetpu_compiler used.

Personally, I built for a coral USB device with edgetpu runtime version 14. If you set to the minimum runtime version to 14 however, you will have a problem because the libedgetpu libraries mentioned above are compiled for runtime version 13--which luckily v14 devices should be backwards compatible for. This means that the tflite binary needs to be compiled with edgetpu_compiler versions 2 or above as seen here.

The versions of the edgetpu runtime and compiler can be checked as follows:

Get the edgetpu runtime version of the device:

println!("{}", edgetpu::version());

Get the edgetpu compiler version:

edgetpu_compiler --version

Note: The edgetpu compiler will not run on a pi. Arm support has been discontinued.

Note: that the prebuilt edgetpu_compiler version from the github repo is only version 1. This means that you will need to either build from source or be using the apt package manager to pull the up-to-date package. For me that was v16.0.384591198 (speak of terrible semantic versioning 384591198 patches).

Note: Your compiler and runtime versions determine what operations you can use. (This is still a problem for me to this day since, for some reason, parts of my models can not be compiled onto the edgetpu requiring a transfer back to the CPU and resulting in somewhat severe latency)

Compilation for the edgetpu is to be done with: (Although I recommend reading through the https://coral.ai/docs)

 edgetpu_compiler -m 13 converted_quantized_model.tflite 

Additionally, you will need to convert your tf models, before compacting or compiling them for the edgetpu, with the tf.compat.v1, if you are not making your models in tensorflow v1.14.

You may also need to use TOCO, the old compacter, setting experimental_new_converter to false. I do not believe the tf.lite.TFLiteConverter from tensorflow stable version 2.4 's experimental converter was capable of quantization and gave really annoying errors. I do believe this has been added in tensorflow r2.5.0 and later.

My models, for reference, were compiled with min runtime v13 edgetpu_compiler v16.0.384591198 and trained with stable tensorflow v2.4. This worked fine for me.

Best Regards,
Luke

from edgetpu-rs.