araobp / image-features Goto Github PK

View Code? Open in Web Editor NEW

Image features for edge AI

License: GNU General Public License v3.0

C 94.85% C++ 0.73% Assembly 0.37% Python 0.17% Batchfile 0.01% Jupyter Notebook 3.88%

image-features's Introduction

image-features

(Work in progress, migrating RGB565 to YUV4:2:2: code in development)

Motivation

I am just interested in computer vision at lower resolutions (32x32 or 128x128) for edge AI.

Limited resources

Since the RAM size of STM32F4 Discovery board is very limited (around 100KBytes), I mainly handle 32x32 resolution images in this project.
Although deep learning is quite a heavy task, it is possible to run deep learing on 32x32 resolution images with a limited number of classes on my PC.

Architecture

                 STM32L4 Discovery
                ...................
                :                 :
 [OV7670]----->[DCMI]        [UASRT]------>[Python program on my PC]
                :                 :
                ...................

CAD

Code

Keras model (Jupyter Notebook)

I captured a hundred of images for each class (rock, paper, scissors) by using the viewer, then trained CNN with the data: Rock-paper-scissors.

Use cases

Rock-paper-scissors

I made X-CUBE-AI/CubeMX to generate image classification code based on the Keras model. The code could classify "rock", "paper" and "scissors".

--- Inference ---
 PAPER         1%
 ROCK         26%
 SCISSORS     71%

--- Inference ---
 PAPER         0%
 ROCK         53%
 SCISSORS     46%

--- Inference ---
 PAPER        54%
 ROCK         13%
 SCISSORS     31%
       :

I honestly admit that the result above was the best case, because the view angle was too small (25 degrees is not enough for hand gesture recognition).

I exchanged a camera with another one that supports 170 degrees view angle. Since the image sensor is not big enough, I could only get around 60 degrees of view. But the inference result was significantly improved.