edgeimpulse / example-esp32-cam Goto Github PK

View Code? Open in Web Editor NEW

82.0 17.0 27.0 7.12 MB

Builds and runs an exported image classification impulse on ESP32 Cam

License: Apache License 2.0

example-esp32-cam's Introduction

[Deprecated] - ESP32 Cam and Edge Impulse

Update 2022/10:

ESP32 examples are now directly available in the Arduino Library deployments. This repository should no longer be used.

Update 2022/04/27:

We officially released the support for the ESP-EYE on April 27th 2022 which uses ESP-IDF instead of Arduino. You can use other boards too by changing the pins definition and recompiling your firmware. This repository is still working but there as there is no hardware acceleration available on the Arduino IDE esp32 board definition, we strongly suggest to use ESP-IDF.

How to run custom inference on a ESP32 cam using Edge Impulse.

Material

This code has been tested the AI Thinker ESP32 Cam module. It should work the same with the Wrover board or an board that has PSRAM.

To use this board, please select your board in the Arduino code the following lines:

// Select camera model

#define CAMERA_MODEL_WROVER_KIT // Has PSRAM
//#define CAMERA_MODEL_ESP_EYE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_PSRAM // Has PSRAM
//#define CAMERA_MODEL_M5STACK_V2_PSRAM // M5Camera version B Has PSRAM
//#define CAMERA_MODEL_M5STACK_WIDE // Has PSRAM
//#define CAMERA_MODEL_M5STACK_ESP32CAM // No PSRAM
//#define CAMERA_MODEL_AI_THINKER // Has PSRAM
//#define CAMERA_MODEL_TTGO_T_JOURNAL // No PSRAM

Steps

Create your Image Classification model using Edge Impulse.

Due to the board limitations, you may need to train your model with 96x96 images and use the MobileNetV1 0.01:

Download the Arduino library under the Deployment tab in the Edge Impulse studio

Basic Image Classification Example

Note: On Sept 9th 2021, the issue predicting always the same classes has been fixed

Open the Basic-Image-Classification.ino file under the /Basic-Image-Classification folder.
Import the .zip library you have downloaded from Edge Impulse Studio
Change the #include <esp32-cam_image-classification_inference.h> line according to your project name.
Compile and deploy the code to your board
Open the serial monitor and use the provided IP to capture an image and run the inference:

Advanced Image Classification Example (Deprecated)

Note: Deprecated since Basic Example has been fixed and also support bilinear interpolation technique to resize the frame.

Note 2: Here we use the ESP SDK to resize the image in RGB888 format using the bilinear interpolation technique. You can see the funtion declaration on Espressif's Github repository.

Open the Basic-Image-Classification.ino file under the /Advanced-Image-Classification folder.
Set your WIFI credentials
Navigate to the app_httpd.cpp tab.
Import the .zip library you have downloaded from Edge Impulse Studio like on the previous example:
Change the #include <euros_coins_classification_inference.h> line according to your project name.
Compile and deploy the code to your board
Open the serial monitor and use the provided IP to capture an image and run the inference:
On your brower navigate to the IP provided by your Serial Console
Use the toggle button to activate the Edge Impulse Inference (you need to select an image resolution lower or equal to QVGA).
Click on Run inference:

Ressources

Note: Theses tutorials / repositories have been used to create this project:

example-esp32-cam's People

Contributors

Stargazers

Watchers

example-esp32-cam's Issues

Unable to train Object Detection with 48x48 image size

It seems that Edge Impulse may have changed something on their side, I am not able to train object detection for images with a size of 48x48, I will get an error. Any idea on when this will change? I really want this to work!

Missing libraries

Looks like the esp-dl (https://github.com/espressif/esp-dl, former esp-face) repo has changed so much that many libraries went missing in this example. My temporary solution is go to this forked repo and copied all the .c and .h files under image_util and lib into the project dir to make it work.

It would be great if you can update it to fit Espressif's new APIs.

missing lib

After following the guide for a simple object detector on the esp32 I hit

vector: No such file or directory
#include

would be great if these examples were actually vetted and working...

problem image_util.h

i can't find this "image_util.h"
The software engineer on the Edge Impluse platform said that his image_util.h file was in the Arduino program file, but I couldn't find it
so anywhere i can find this file

Esp32 CAM Serial Connection issue with edge impulse deamon cli

I am using esp32 cam module but facing issue in connecting serial with device and edge imple cli.
It is giving an error of timeout.
I want to understand is esp32 cam module is supported by edge impulse if not then how it is different from esp eye.

FRAMESIZE_240X240

Advanced-Image-Classification:53:25: error: 'FRAMESIZE_240X240' was not declared in this scope
     config.frame_size = FRAMESIZE_240X240;
                         ^
Advanced-Image-Classification:57:25: error: 'FRAMESIZE_240X240' was not declared in this scope
     config.frame_size = FRAMESIZE_240X240;
                         ^
Advanced-Image-Classification:71:23: error: 'FRAMESIZE_240X240' was not declared in this scope
   s->set_framesize(s, FRAMESIZE_240X240);

basic and advanced examples not compiling

Memory allocation error when deploying example model

Hello,

We are trying to deploy the example model from the tutorial from this GitHub repository README (https://www.survivingwithandroid.com/tinyml-esp32-cam-edge-image-classification-with-edge-impulse/)
on our ESP-EYE device.

When using the most basic model (MobileNetV2 96x96 0.05) in Edge-Impulse the deployment works but the model is not accurate. Every other model fails with the following errors:

When deploying the model with the default partitions scheme we are getting the following error:
WiFi connected\n
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://192.168.1.158' to connect
Capture image
Edge Impulse standalone inferencing (Arduino)
ERR: Failed to run DSP process (-1002)
run_classifier returned: -5
When deploying the model in arduino IDE using the "Huge APP" partition scheme we are getting the following error:
WiFi connected
Starting web server on port: '80'
Starting stream server on port: '81'
Camera Ready! Use 'http://192.168.1.158' to connect
Capture image
Edge Impulse standalone inferencing (Arduino)
ERR: failed to allocate tensor arena
Failed to allocate TFLite arena (error code 1)
run_classifier returned: -6

The ESP-EYE has 4MB of memory available.
According to the arduino IDE, the code itself takes ~1.2MB of memory.
According to the Edge-Impulse website, all models do not need more than 1MB of additional memory. However, it seems that the memory is the issue here.

Adding a screenshot of our board settings in arduino IDE:

Can you please advise on how can we make the more complicated models work on our device?
Thank you!

Curious about output prompt : dsp vs classification

Hi,
Thank you for your wonderful example about ESP32 board!

But I'm curious about the difference between DSP and Classification section.
The Photo are below.

I'm using the ESP-EYE board.
When I used another board, when I used DSP, I got 0ms in Classification.
The ESP32 board doesn't appear to have a DSP module, but there's a DSP in the results window.

Regards,