Code Monkey home page Code Monkey logo

simpnet_keras_r's Introduction

SimpNet in Keras for R

This is a repository for translating SimpNet in an R flavored Keras implementation. SimpNet is a deep convolutional neural network architecture reported on in:

Towards Principled Design of Deep Convolutional Networks: Introducing SimpNet
Seyyed Hossein Hasanpour, Mohammad Rouhani, Mohsen Fayyaz, Mohammad Sabokrou and Ehsan Adeli

arXiv pre-print: Hasanpour et al. 2018 on arXiv

This is a link to the original SimpNet Github repository

From the author's abstract:

We empirically show that SimpNet provides a good trade-off between the computation/memory efficiency and the accuracy solely based on these primitive but crucial principles. SimpNet outperforms the deeper and more complex architectures such as VGGNet, ResNet, WideResidualNet \etc, on several well-known benchmarks, while having 2 to 25 times fewer number of parameters and operations. We obtain state-of-the-art results (in terms of a balance between the accuracy and the number of involved parameters) on standard data sets, such as CIFAR10, CIFAR100, MNIST and SVHN.

Schematic SimpNet architecture (Hasanpour et al. 2018)

Schematic SimpNet architecture (Hasanpour et al. 2018)

Keras Version

This repo is an attempt to translate the SimpNet architecture into the Keras API via R and the Rstudio flavor of Keras.

So far I have translated:

* SimpNetV2 for MNIST
    * MNIST_SimpleNet_GP_13L_drpall_5Mil_66_maxdrp
* SimpNetV2 for CIFAR10
    * CIFAR10_SimpleNet_GP_13L_drpall_8Mil_66_DRP_After_Pooling

Results of Keras version

For the original results of SimpNet in its native Caffe implementation, see the article and GH repo. The results below are my translation and may not be fully reflective of the native implementation. I will keep trying to match the published results.

MNIST version

Epochs: 10
Test Loss: 0.01990014
Test Accuracy: 0.9949

CIFAR10 version

CIFAR10 results by Hasanpour et al. 2018 are 95.89, I will continue to work on this one.

Epochs: 50
Test Loss: 0.5701714
Test Accuracy: 0.887

Training history for CIFAR10

Training history for CIFAR10

simpnet_keras_r's People

Contributors

mrecos avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Forkers

systats

simpnet_keras_r's Issues

Here are some notes for a better port

First of all thank you for your effort on porting SimpNet.
I had a quick look at your scripts, and noticed there were several pieces missing.
For CIFAR10/100, you need to preprocess the dataset. normalize and zero-pad them. You can have a look at this script here .
Then you can use horizontal flipping and start your training (with random crops of 32x32 as is done in Caffe)
You also need to pay attention to the weight decay and the way the optimization is carried out. As for the 8.9M experiment, I guess I used these parameters:

base_lr: 0.1
lr_policy: "multistep"
gamma: 0.1
momentum: 0.9
weight_decay: 0.005
delta: 0.001
stepvalue: 50000
stepvalue: 95000
stepvalue: 153000
stepvalue: 195000
stepvalue: 220000
stepvalue: 270000
type: "AdaDelta"

You can also lower the weight decay to sth like 0.0005 but increase the dropout ratio e.g. for the layers that have more neurons (i.e the last 3,4 layers), (this will be more effective when you have a warm start).
I'm dont know Keras, so in porting different hyper parameters from Caffe to Keras, I cant really help, anything else, please feel free to contact me (preferably through email : Coderx7 at (Gmail.com)

Good luck

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.