margaretmz / cartoonizer-with-tflite Goto Github PK

I am using tf2onnx to convert the pb file to onnx
!python -m tf2onnx.convert --saved-model /content/saved_model_dir/ --output model.onnx --opset 10 to get a onnx file but get this error when parsing
The ONNX model's slice layer input const_slice__92 must be a 1-dimensional constant tensor

java.io.FileNotFoundException: File doesn't exist: /data/app/com.tflite.whitebox_cartoon_gan-y_KbloIgyy3HoRZROCLR8w==/base.apk

@khanhlvg here's how I prepared the post-processing stats for populating metadata:

# Creates output info, cartoonized image
    output_image_meta = _metadata_fb.TensorMetadataT()
    output_image_meta.name = "cartoonized_image"
    output_image_meta.description = "Image cartoonized."
    output_image_meta.content = _metadata_fb.ContentT()
    output_image_meta.content.contentProperties = _metadata_fb.ImagePropertiesT()
    output_image_meta.content.contentProperties.colorSpace = (
        _metadata_fb.ColorSpaceType.RGB)
    output_image_meta.content.contentPropertiesType = (
        _metadata_fb.ContentProperties.ImageProperties)
    # output_image_normalization = _metadata_fb.ProcessUnitT()
    # output_image_normalization.optionsType = (
    #     _metadata_fb.ProcessUnitOptions.NormalizationOptions)
    # output_image_normalization.options = _metadata_fb.NormalizationOptionsT()
    # output_image_normalization.options.mean = [0.0]
    # output_image_normalization.options.std = [0.003921568627]  # 1/255
    # output_image_meta.processUnits = [output_image_normalization]
    output_image_stats = _metadata_fb.StatsT()
    output_image_stats.max = [255.0]
    output_image_stats.min = [0.0]
    output_image_meta.stats = output_image_stats

What would be the best way to account for the fact that the output needs to be scaled to [0,255]?

when i capture image from camera then its prccessing but after that its crash

I figured out the (black) image issue on Android! It was caused by incorrect image post processing (from tflite metadata).

@sayakpaul please fix the metadata in the tflite models as follows:
update from
output_image_normalization.options.mean = [0.0] output_image_normalization.options.std = [1.0]
to
output_image_normalization.options.mean = [-1]. output_image_normalization.options.std = [0.00784313] # 1/127.5

I have verified that this will mirror the post processing in the conversion notebook, and it works on Android.

cc: @khanhlvg

When adding metadata to the tflite models, I also find it helpful to put a date stamp in the version code for example something like this: v3_2020-08-03. Otherwise it's really difficult for me to keep track of the various model versions.

This is not the issue but there must be c++ code as well
Kindly upload c++ code with opencv using c API of tflite

Under the ml folder, add a markdown witha bit info about the tflite models. Just need links to models and any necessary barely minimum info. We can wait till after tutorial is complete to make it elaborative.

Need to use OpenCV for converting between BGR and RGB, in data pre & post processing on Android.

@margaretmz @khanhlvg

As per our discussion here we decided to go with fixed-shaped TFLite models (520x520). Also, the models are not metadata populated yet which also came as one of the decisions in the aforementioned thread. With better model binding support in the coming days, we can release v2 of these models. WDYT?

All the models are temporarily hosted on a GCS bucket. I have included the links here.

I have also included a fresh notebook to demo the conversion and Python inference process.

Let me know what y'all think about the TF Hub publication step from here.

As GAN model usually consume a lot of memory, we couldn't use them on large images. I wonder if we can convert a small image to cartoon (e.g. 256 * 256) then use ESRGAN to enlarge it (e.g. to 512 * 512).

@sayakpaul
Could you be able to try the approach on a notebook and see the quality of it compared to directly convert a 512 * 512 images to cartoon?

@margaretmz FYI