ml-opt / dnn_opt Goto Github PK
View Code? Open in Web Editor NEWdnn_opt is a machine learning library to perform optimization and neural network training by using meta-heuristics with multi-core and GPU support
dnn_opt is a machine learning library to perform optimization and neural network training by using meta-heuristics with multi-core and GPU support
We need to create a notebook to compare gradient based training of MLP networks with meta-heuristic training. We suggest to proceed as follows:
The output of this issue should be a new folder within docs
directory.
docs/bench-accuracy
mlp-sgd-vs-meta.ipynb
and mlp-sgd-vs-meta.html
notebook.This issue is a continuation of issue #10.
We need to implement the following test functions in parallel using CUDA, that is in cuda
layer:
Each function specification can be found in: http://arxiv.org/abs/1308.4008v1
The scope of this issue is parallel implementation using CUDA in the cuda
layer.
Important: remember to modify the solution folder structure according to the modifications done in #6 - #7 (create bench folder and namespace).
We need to create a notebook comparing the efficiency, speed-up and scalability of sequential and OMP implementations of the standard benchmark functions. For this comparison, we should consider the following benchmark functions:
In addition, consider to perform the experiments using different solution vector sizes (from
This output of this issue should be a new folder within docs directory.
Provide a sequential implementation for the algorithm Gray Wolf Optimization introduced by:
MIRJALILI, Seyedali; MIRJALILI, Seyed Mohammad; LEWIS, Andrew. Grey wolf optimizer. Advances in engineering software, 2014, vol. 69, p. 46-61.
I want to implement a series of test functions to validate the performance of the optimization algorithms used in the dnn_opt library.
This issue is a continuation of issue #6.
We need to implement the following test functions in parallel using OMP, that is in copt
layer:
Each function specification can be found in: http://arxiv.org/abs/1308.4008v1
The scope of this issue is parallel implementation using OMP in the copt
layer.
Important: remember to modify the solution folder structure according to the modifications done in #6 - #7 (create bench folder and namespace).
csv
as this will be more intuitive for usersseparator
We need to include a set of benchmark datasets to compare the accuracy and efficiency of our neural implementations. In this case, we are looking for small-size datasets (less that 1k examples) that deals either classification or regression. Probably, a good starting point are the following publicly available datasets:
We need to create a notebook transforming the data as follows:
Every steep should be documented and explained in the notebook. Include in the notebook the following information for each dataset:
The output of this issue should be a new directory structure within docs
folder. Create the following hierarchy of folders:
docs/bench_data/small/original/
: with all the .ARFF original files.docs/bench_data/small/transformed/
: with all the transformed files.docs/bench_data/small/notebooks:
with the transform.ipynb
and transform.html
notebook that can be used to generate the docs/bench_data/small/transformed/
files from the files in docs/bench_data/small/original/
.Provide a member function to set the seed of any generator
Gradient based algorithms are the default training algorithms for ANN. Hence, providing support for such algorithms (SGD, ADAM, RMSProp, etc.) is critical in order to provide out of the box benchmarking capabilities. Our suggestion is to proceed as follows:
Create a new class in base
package (e.g. class derivable : public solution {}
that will inherit from solution class
). derivable
should have an array of floats (e.g. derivable::m_df
) that represents the derivative of the solution::fitness()
function respect each parameter in solution::get_params()
. Hence, derivable::get_df()
size will be solution::size()
.
Define a getter in derivable
class (e.g. derivable:df()
) that, in case the solution was modified will calculate the derivative of the fitness function and store the result in derivable::m_df
array. In case the solution is not modified, it will simply return derivable::m_df
. The implementation of this method should be the same as in the current solution::fitness()
method.
As in the case of solution::fitness()
and solution::calculate_fitness()
consider an implementation of derivable:df()
and a protected virtual derivable:calculate_df() = 0
method in derivable
. It is probably a good idea to do not create derivable::m_df
before the first derivable:df()
call just in case the derivative gets never used.
Each child of derivable
in solutions
package should re-implement its own version of virtual derivable:calculate_df() = 0
according to the fitness function (only if the fitness function is derivable of course). This means that the network
class should inherit from derivable
instead of solution
and implement virtual derivable::calculate_df() = 0
.
The network:calculate_df()
implementation will call a layer::backprop()
method defined in the layer class, passing the position in the derivable::m_df
array where the layer will store the derivative of its corresponding parameters. layer::backprop()
method should be similar to the current layer::prop()
method.
Each child of layer
in package layers
should re-implement its own version of virtual layer:backprop() = 0
. Currently there should be a single layer fc
(fully connected layers) implemented in the library.
Create a new class in algorithms
package class sgd : public algorithm
that using the derivative and the fitness function of a derivable
solution can implement Stochastic Gradient Descent Algorithm.
Include a reader classes to support input files in several standard formats. For the moment, it would be useful to consider:
Please update the list as you implement the reader for each format. In addition, include web links to a description of each format as you proceed. I would like to see a separated comment clarifying how each format was implemented if needed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.