machine-learning-tokyo / dl-workshop-series Goto Github PK

View Code? Open in Web Editor NEW

935.0 67.0 253.0 15.34 MB

Material used for Deep Learning related workshops for Machine Learning Tokyo (MLT)

License: Apache License 2.0

Jupyter Notebook 100.00%

deep-learning workshop convolutional-neural-networks cnn-keras keras

dl-workshop-series's Introduction

DL-workshop-series

Code used for Deep Learning related workshops for Machine Learning Tokyo (MLT)

Part I: Convolution Operations

Implementation

ConvKernels: colab notebook with simple examples of various kernels applied on an image using convolution operation ConvNets: colab notebook with functions for constructing keras models. Models:

AlexNet
VGG
Inception
MobileNet
ShuffleNet
ResNet
DenseNet
Xception
Unet
SqueezeNet
YOLO
RefineNet

Slides

Link to the presentation: https://drive.google.com/open?id=1sXztx3E9M3G0BIRLh6sxaqVOEOdJVJTrzHOixA5b-rM

Cheat Sheet:

Video series: CNN Architectures (including implementation)

Part II: Learning in Deep Networks

dl-workshop-series's People

Contributors

Stargazers

Watchers

Forkers

zehrahayirci manoja328 hal2001 rubenszimbres lazarusa shahnawazgrewal cuulee tspannhw stoney1 garrett-cn ayepete laszlokiraly barbecacov tabshaikh amirunpri2018 saugat-prisma marcelomata fadeyklux hack121 geauxapp batermj priyakansal ericksonrn andreacrescini onisimchukv orrimoch randomwalker18 giserh kartheekpnsn 5trobl barseghyanartur jacekwachowiak jhgil92 jaedukseo sibearradio thorpham jafffy fendaq xing-long studian iamtpb bbruhh htnani matekristof dexter1618 shankar0206 karanr93 vandread12 pseudobublar oskop hs99 ngoduyvu daniellsm triper1022 iamharshverma hunglethanh9 mutum dashankadesilva yoavramon escapist-007 robintux dbaruajuniv arjunfzk abdulbasitkhaleeq mbrukman soramaruyama suzana-ilic shreyanshp saq1410 radovankavicky gapdata shreya1sharma shekharrajak codeaudit vatsaljrathod shanliwa1 locussam sniafas virgi1io dhrg michaelbroox omarzaghlol linhlpv alishakiba ssgalitsky ibukunchife vinod-koli lanhhv84 mnatsirm iamshivamjaiswal reedip ersawant albertofernandezvillan juliac29 bilaldendani minuteswithmetrics sprinterzzj qoboty snazari aaravrav142

dl-workshop-series's Issues

Densenet is not right

ResNet downsampling @ ConvNets.ipynb

Thanks for sharing.
But it seems that downsampling should be performed by stride-2 convolutions in the 3×3
layer of the first block in each stage. This way, number of parameter may be reduced.

def resnet(input_shape, n_classes):

    def conv_bn_rl(x, f, k=1, s=1, p='same'):
        x = Conv2D(f, k, strides=s, padding=p)(x)
        x = BatchNormalization()(x)
        x = ReLU()(x)
        return x

    def conv_block(tensor, f1, f2, s):
#         x = conv_bn_rl(tensor, f1, s=s)
#         x = conv_bn_rl(x, f1, 3)
        x = conv_bn_rl(tensor, f1)
        x = conv_bn_rl(x, f1, 3, s=s)
        x = Conv2D(f2, 1)(x)
        x = BatchNormalization()(x)

        shortcut = Conv2D(f2, 1, strides=s, padding='same')(tensor)
        shortcut = BatchNormalization()(shortcut)

        x = add([shortcut, x])
        output = ReLU()(x)

        return output

according to Aggregated Residual Transformations for Deep Neural Networks

Some Colab links may be broken

E.g. in https://github.com/Machine-Learning-Tokyo/DL-workshop-series/blob/master/Part%20I%20-%20Convolution%20Operations/ConvKernels.ipynb

Notebook not found
There was an error loading this notebook. Ensure that the file is accessible and try again.
Ensure that you have permission to view this notebook in GitHub and authorize Colaboratory to use the GitHub API.

https://github.com/Machine-Learning-Tokyo/DL-workshop-series/blob/master/ConvKernels.ipynb

RefineNet in ConvNets.ipynb

In RefineNet of the ConvNets.ipynb notebook, function rcu residual convolution unit. Code:

x = ReLU()(tensor)
x = Conv2D(f, 3, padding='same')(x)
x = ReLU()(tensor)
x = Conv2D(f, 3, padding='same')(x)

maybe should be replaced by:

x = ReLU()(tensor)
x = Conv2D(f, 3, padding='same')(x)
x = ReLU()(x)
x = Conv2D(f, 3, padding='same')(x)

i.e. tensor in the second ReLU should be replaced by x. Otherwise, the output of the first ReLU+Conv2D is rewritten by the second ReLU+Conv2D. This assumption is confirmed by the article, where RCU (Residual Conv Unit) has sequential ReLU->Conv2D->ReLU->Conv2D.

machine-learning-tokyo / dl-workshop-series Goto Github PK

dl-workshop-series's Introduction

DL-workshop-series

Part I: Convolution Operations

Implementation

Slides

Video series: CNN Architectures (including implementation)

Part II: Learning in Deep Networks

dl-workshop-series's People

Contributors

Stargazers

Watchers

Forkers

dl-workshop-series's Issues

Densenet is not right

ResNet downsampling @ ConvNets.ipynb

Some Colab links may be broken

RefineNet in ConvNets.ipynb

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent