Code Monkey home page Code Monkey logo

niy's Introduction

A super mini but powerful deep learning framework written in pure c language

Windows Linux

Due to a lack of time, this project is closed

Installation

Select one of the following experiments

Or try some funny examples:

Why Niy?

  • Learn without writing any code
  • Super easy
  • Super mini
  • Better performance over a variety of tasks

Supported Features

Layers

  • Convolution Layer
    • Grouped Convolution Layer
    • Dilated (Grouped) Convolution Layer
    • Locally Connected (Grouped) Convolution Layer
    • Locally Connected Dilated (Grouped) Convolution Layer
  • Deconvolution Layer
    • Grouped Deconvolution Layer
    • Dilated (Grouped) Deconvolution Layer
    • Locally Connected (Grouped) Deconvolution Layer
    • Locally Connected Dilated (Grouped) Deconvolution Layer
  • Pooling Layer
    • Max Pooling
    • Abs Max Pooling
    • Average Pooling
    • Random Pooling
  • Merge Layer
  • Dropout Layer
  • Shortcut Layer

Example

Let's take the famous mnist dataset as example

Step 1: Edit model file

# model.txt
[
    {Shape: [28, 28, 1]},
    {Shape: [14, 14, 2], Kernel: [6, 6], Pad: [2, 2]},
    {Shape: [7, 7, 4], Kernel: [6, 6], Pad: [2, 2]},
    {Shape: [1, 1, 10], Kernel: [7, 7]},
]

Step 2: Edit conf file

# conf.txt
{
    Load: 0,
    
    RateInit: 0.02,

    EpochTrain: 8,
    EpochDecay: 0.2,
    EpochLog: 0.2,

    InputType: 'pointwise',
    LossType: 'max',
    Regularization: 2,

    Image: {
      Draw: true,
      Src: 'layer 0',
    },

    FreqTest: 2,
}

Step 3: Run

# niy conf.txt

You will get a test accuracy around 98.5% after a few minutes

Tutorial

There are four kinds of file

  1. Model
     Stores model information
  2. Conf
     Stores configuration information
  3. Sample
     Stores sample data
  4. Para
     Stores biases, weights, loss, rate...

Niy uses CON to define model and conf files

Model

Name Description Default
Shape layer shape [0, 0, 0]
Reshape change the layer shape when output null
Kernel kernel size [0, 0]
Dilation dilate the kernel 1
Pad padding [0, 0]
PoolType 'max'
'ave'
'amax': absolute maximum
'rand'
'ave'
IsLocal switch to locally connected layer false
IsReverse switch to deconvolution layer false
IsMerge switch to merge layer 1
IsFar switch to far convolution/deconvolution layer false
Group split the channels into groups 1
Dropout dropout null
Shortcut shortcut to one top layer null
Offset channel offset for shortcut layer 0
IsContinuous whether or not to connect to the previous layer, used in multi-branch network true

Conf

Name Description Default
Debug show debug information or not true
Load 0: do not load
1: load biases and weights only
2: load all
0
RateInit initial learning rate 0.02
RateDecay learning rate decay 0.9 
InputType 'pointwise'
'onehot'
'pointwise'
LossType 'mse'
'softmax'
'max'
'zero'
'mse'
LossInit initial loss
LossMul used to calculate the moving average of loss 0.001 
Regularization used to limit bias/weight
BiasFiller.Type 'zero', 'uniform', 'usni', 'usno', 'usnio', 'usnf', 'gaussian', 'gsni', 'gsno', 'gsnio', 'gsnf'
"u": uniform, "g": gaussian, "s": sqrt, "n": number, "i": input, "o": output
'zero' 
BiasFiller.Adj used to adjust initial bias value
new = (old + adj[0])*adj[1]
[0, 1] 
WeightFiller.Type 'zero', 'uniform', 'usni', 'usno', 'usnio', 'usnf', 'gaussian', 'gsni', 'gsno', 'gsnio', 'gsnf'
"u": uniform, "g": gaussian, "s": sqrt, "n": number, "i": input, "o": output
'gsni' 
WeightFiller.Adj used to adjust initial weight value
new = (old + adj[0])*adj[1]
[0, 0.8] 
Image.Draw draw current image or not false 
Image.Adj used to adjust pixel value
new = (old + adj[0])*adj[1]
[0.5, 255]
Image.Src 'layer 0'
'layer last'
'predict'
'layer 0'
Image.Path image file path '_img.bmp'
Model model file path 'model.txt'
Para para file path 'para.bin'
TrainIn train input sample file path 'train_in.smpl'
TrainOut train output sample file path 'train_out.smpl'
TestIn test input sample file path 'test_in.smpl'
TestOut test output sample file path 'test_out.smpl'
PredictIn predict input sample file path 'predict_in.smpl'
PredictOut predict output sample file path 'predict_out.smpl'
EpochTrain how many epochs to train
EpochDecay how often to decay 0.2 
EpochLog how often to log 0.2 
FreqTest the frequency to test
0: never
1: test after training
2: test at each log time
FreqPredict the frequency to predict
0: never
1: predict after training
2: predict at each log time
FreqSave the frequency to save
0: never
1: save after training
2: save at each log time
TestTable test table size
PredictTable predict table size

Sample

Address Datatype Description
0 int32 set to zero 
4 int32 sample shape X-axis 
8 int32 sample shape Y-axis 
12 int32 sample shape Z-axis 
16 int32 number of samples
if set to zero, the program will calculate automatically
20 int32 image shape X-axis
set to zero if not used
24 int32 image shape Y-axis
set to zero if not used
28 int32 image shape Z-axis
set to zero if not used
32 float32 array sample data

TODO

  • Support GPU
  • Support Intel MKL
  • Support RNN/LSTM
  • Support Deep Reinforcement Learning

License

Apache License 2.0

niy's People

Contributors

microic avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.