• Updated uTensor ReadMe
• Updated uTensor-CLI ReadMe
• Added Contributor Guide
• Added new uTensor project guide
• Dropout Support

uTensor is an extremely light-weight machine learning inference framework built on Mbed and Tensorflow. The project contains a runtime library and an offline tool. The total size of graph definition and algorithm implementation of a 3-layer MLP produced by uTensor is less than 32kB in the resulting binary (excluding the weights).

A model is constructed and trained in Tensorflow. uTensor takes the model and produces a .cpp and .hpp file. These files contains the generated C++11 code needed for inferencing. Working with uTensor on the embedded side is as easy as copy-and-paste. The function interface looks like this:

#include "models/deep_mlp.hpp"
...
Context ctx;  //creating a context
...
//preparing for the input tensor
...
get_deep_mlp_ctx(Context& ctx, Tensor* input_0);  //perform inference
ctx.eval();
S_TENSOR prediction = ctx.get({"y_pred:0"});  //getting the result

The .hpp and .cpp files can be generated given a model (protocal buffer) file, for example:

$ utensor-cli deep_mlp.pb --output-nodes=y_pred
...
... Generate header file: models/deep_mlp.hpp
... Generate source file: models/deep_mlp.cpp

The project is work-in-progress. Here are the operators, of their quantized versions, that are currently avaliable:

• Add
• ArgMax
• Dropout
• MatMal
• Max
• Min
• Placeholder
• Quantization Ops
• ReLu
• Reshape

uTensor should support any Mbed enabled board that has sufficient memory (128+ kB RAM and 512kB+ flash recommended). However, these two boards are popular among the core developers:

• DISCO-F413ZH: a good demo/application prototyping platform, wi-fi
• K64F: rock-solid development environment

You will need a FAT32 formated SD card. Please note, the size of the SD card has to be less than 32GB. An SD card will be made optional in the future releases.

There are two flows to get started with uTensor. For Windows users, please choose the Cloud9 flow as shown below.

• The Cloud9 Flow
  • Requires Amazon Cloud9 Account
  • Does not support runtime debugging
• Local Installation
  • Requires Mbed-CLI installation (Python 2)
  • Requires uTensor-CLI installation

The example uses a 3-layer MLP trained on the MNIST dataset. The touch screen input is fed into the neural network for processing and the result is printed on the screen.

This example shows how to buffer time-series data into batches of snapshots. These snapshots are then fed into the neural network for inferencing. The model a small multi-layer MLP trained on the ADL dataset.

Please refer to this guide for instructions on creating your own project from stretch on DISCO-F413ZH.

uTensor is young and under going rapid development. Many exciting features are on the way:

• Convolution
• Pooling
• SD cards optional: ability to store weights in on-board flash
• CMSIS-NN integration
• Smaller binary

You can also check the project page for the latest progress. If you'd like to take part in this project, please have a look at our contributor guide and feel free to reach out to us.

• Why Edge Computing
• Why the Future of Machine Learning is Tiny
• TensorFlow
• Mbed
• Node-Viewer
• How to Quantize Neural Networks with TensorFlow
• mxnet Handwritten Digit Recognition