tiny-cnn: A C++11 implementation of deep learning (convolutional neural networks)
tiny-cnn is a C++11 implementation of deep learning (convolutional neural networks).
design principle
- fast, without GPU
- with TBB threading and SSE/AVX vectorization
- 98.8% accuracy on MNIST in 13 minutes training (@Core i7-3520M)
- header only, policy-based design
- small dependency & simple implementation
supported networks
layer-types
- fully-connected layer
- fully-connected layer (with dropout)
- convolutional layer
- average pooling layer
activation functions
- tanh
- sigmoid
- rectified linear
- identity
loss functions
- cross-entropy
- mean-squared-error
optimization algorithm
- stochastic gradient descent (with/without L2 normalization)
- stochastic gradient levenberg marquardt
dependencies
- boost C++ library
- Intel TBB
sample code
construct convolutional neural networks
#include "tiny_cnn.h"
using namespace tiny_cnn;
using namespace tiny_cnn::activation;
void cunstruct_cnn() {
using namespace tiny_cnn;
// specify loss-function and optimization-algorithm
typedef network<mse, gradient_descent> CNN;
CNN mynet;
// tanh, 32x32 input, 5x5 window, 1-6 feature-maps convolution
convolutional_layer<CNN, tan_h> C1(32, 32, 5, 1, 6);
// tanh, 28x28 input, 6 feature-maps, 2x2 subsampling
average_pooling_layer<CNN, tan_h> S2(28, 28, 6, 2);
// fully-connected layers
fully_connected_layer<CNN, sigmoid> F3(14 * 14 * 6, 120);
fully_connected_layer<CNN, identity> F4(120, 10);
// connect all
mynet.add(&C1); mynet.add(&S2); mynet.add(&F3); mynet.add(&F4);
assert(mynet.in_dim() == 32 * 32);
assert(mynet.out_dim() == 10);
}construct multi-layer perceptron(mlp)
#include "tiny_cnn.h"
using namespace tiny_cnn;
using namespace tiny_cnn::activation;
void cunstruct_mlp() {
typedef network<mse, gradient_descent> MLP;
MLP mynet;
fully_connected_layer<MLP, sigmoid> F1(32 * 32, 300);
fully_connected_layer<MLP, identity> F2(300, 10);
mynet.add(&F1); mynet.add(&F2);
assert(mynet.in_dim() == 32 * 32);
assert(mynet.out_dim() == 10);
}another way to construct mlp
#include "tiny_cnn.h"
using namespace tiny_cnn;
using namespace tiny_cnn::activation;
void cunstruct_mlp() {
auto mynet = make_mlp<mse, gradient_descent, tan_h>({ 32 * 32, 300, 10 });
assert(mynet.in_dim() == 32 * 32);
assert(mynet.out_dim() == 10);
}more sample, read main.cpp
build sample program
gcc(4.6~)
without tbb
./waf configure --BOOST_ROOT=your-boost-root
./waf build
with tbb
./waf configure --TBB --TBB_ROOT=your-tbb-root --BOOST_ROOT=your-boost-root
./waf build
with tbb and SSE/AVX
./waf configure --AVX --TBB --TBB_ROOT=your-tbb-root --BOOST_ROOT=your-boost-root
./waf build
./waf configure --SSE --TBB --TBB_ROOT=your-tbb-root --BOOST_ROOT=your-boost-root
./waf build
or edit inlude/config.h to customize default behavior.
vc(2012~)
open vc/tiny_cnn.sln and build in release mode.
license
The BSD 3-Clause License
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent