chenditc / deeplearningresearch Goto Github PK

Model Loader should be able to load and existing trained model or a empty model. Requirement:

1. Create a Database that store the parameter of models. Eg. If it's mlp, it should store the layer number, the matrix, and the computation formula.
2. Each model should have a function called: LoadModel(key). This should be able to load the from database and perform prediction.
3. Each model should have a function called: SaveModel(key). This should be able to serialize the parameter to database.
4. Each model should have a function called InitModel(key). This should be able to random initialize the parameter and register it on database.

Training the data should be:
./train --model mlp --data macd --key mlp_macd2

1. This should automatically read the training data from database and all rows should have key macd.
2. The parameters for each layer should also be stored in the database, so we can restore the model later.
3. The model should self contain all the parameter we want to tweak, like, alpha, patience, and etc.

Testing the data should be:
./test --model --key mlp_macd2 [--doc test_data]

1. If no test_data specified, it should read from stdin
2. If test_data specified, it should read the test_data and output a list of prediction

Add a new module to Learning model called MultilayerPerceptron.

1. The layer should be able to change as configuration changes
2. The number of neural for each layer should be able to change.

Rewrite logistic regression model to use DataLoader interface to load data.

Upload standard MNIST data set to database.

1. Upload all stock related data as input (price, macd, etc)
2. Upload stock change as output (up/ down)
3. Use logistic regression to predict it, and see how it works

To train a model, we need to specify a list of parameters, like number of epoch, mini-batch size, patient, L1 L2 regularization. So we want to have a trainer class:

1. Have a method called Load(key) that load all parameters like number of epoch, mini-batch size, patient, L1 L2 regularization.
2. Have a method called InitTrainer() this will initialize a set of training parameters, it can register it to database or not.
3. Have a method called trainModel(modelInstance). This will take the model and train it with necessary steps and call saveModel to save the model after each best approach.

Use multi-thread and queue to increase the speed of upload and download the data.

1. For downloaded data, there should be a queue that keep fetching database to full it self up. This should happen in a separate thread, maybe a separate class
2. For upload data, there should be a queue to do this too.

This has lower priority than the model implementation.

Add a model called ConvolutionalNN:

1. We should be able to specify how many convolutional layer we want to have
2. We should be able to specify how large is each layer.
3. We should be able to specify the scan window

Requirement:

1. Have one database that contains the data for training. Each table contains one category of data. Eg. Macd, Mav, etc.
2. Have a python module:
   a) load_data() Take data set name (table name), data key (version of data), and the number of data entry needed. Return a matrix of data. Each row is one training entry.
   b) *get_data_parameter() Get data's metadata, Eg. dimensionality, mean, std and etc.

In RBM: We must not compute the gradient through the gibbs sampling? Why? How does the probability part looks in gradient decent?

To pre-processing the data, we need to know some meta data, e.g. dimensionality, mean, std and etc.
It will be computational expensive to compute them when training happens. So we need some process to pre-compute meta data and keep consistent as data set updated.

b) *get_data_parameter() Get data's metadata, Eg. dimensionality, mean, std and etc.

The online tester should be able to load the model and use the model to compute the prediction on fly. So it should take 1. model name, 2. parameter name 3. input from stdin and print prediction to stdout

Understand how scan are used in different models. Including RNN, RBM and etc.

Add a new Module Called RecurrentNN:

1. We should be able to specify the number of unit in the recurrent layer
2. We should be able to specify the layer of recurrent layer.

ps. I think we can reuse the multilayer perceptron for this, just link the output and input together