A demo for running quantized CLIP model (ViT-B/32) on Android.

Run this jupyter notebook to get the quantized models:

• clip-image-encoder-quant-int8
• clip-text-encoder-quant-int8

Place them into app\src\main\assets.

Then build and run in your IDE.

Note: Do NOT use PyTorch > 1.13 or it will failed when converting to ONNX format.

This project is just for testing, so forgive my casual code. Good luck :)

• Original (Float 32)
  • ImageEncoder: 335 MB
  • TextEncoder: 242 MB
• Quantized (Int8)
  • ImageEncoder: 91.2 MB
  • TextEncoder: 61.3 MB

Accuracy compared to original CLIP ViT-B/32 model:

Encode 500 pics in single thread:

Device: Xiaomi 12S @ Snapdragon 8+ Gen 1

Note:

• The encode time for each image is 35~45ms
• For images with larger on-disk size, it takes more time to read the image. I have tried ''down-sample'' the large image instead of reading the whole file.

• openai/CLIP
• ONNX Runtime
• mazzzystar/Queryable