The accuracy seems to be worst than with Scikit-learn.

# coding: utf8

from sklearn.metrics import accuracy_score
from sklearn.ensemble import RandomForestClassifier

def do_RandomForest(x_train, y_train, x_test, y_test, n=1000):
  rf = RandomForestClassifier(n_estimators=n)
  rf.fit(x_train, y_train)
  score = rf.score(x_test, y_test)
  y_predict = rf.predict(x_test)
  return score, y_predict

if __name__ == "__main__":
    x_train = [[0,-1], [1,0], [1,1], [1,-1], [2,0], [2,1], [2,-1], [3,2], [0,4], [1,3], [1,4], [1,5], [2,3], [2,4], [2,5], [3,4], [1, 10], [1, 12], [2, 10], [2,11], [2, 14], [3, 11]]
    y_train = [0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2]
    x_test = [[0, -2], [1, 0.5], [1.5, -1], [1, 2.5], [2, 3.5], [1.5, 4], [1, 10.5], [2.5, 10.5], [2, 11.5]]
    y_test = [0, 0, 0, 1, 1, 1, 2, 2, 2]
    score, y_star = do_RandomForest(x_train, y_train, x_test, y_test)
    print score
    print y_star
    # output : 
    # 1.0 
    # [0 0 0 1 1 1 2 2 2]

const {Matrix} = require('ml-matrix');
const {RandomForestClassifier} = require('ml-random-forest');

// our training set (X,Y)
var X = new Matrix([[0,-1], [1,0], [1,1], [1,-1], [2,0], [2,1], [2,-1], [3,2], [0,4], [1,3], [1,4], [1,5], [2,3], [2,4], [2,5], [3,4], [1, 10], [1, 12], [2, 10], [2,11], [2, 14], [3, 11]]);
var Y = Matrix.columnVector([0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2]);

// the test set (Xtest, Ytest)
var Xtest = new Matrix([[0, -2], [1, 0.5], [1.5, -1], [1, 2.5], [2, 3.5], [1.5, 4], [1, 10.5], [2.5, 10.5], [2, 11.5]])
var Ytest = Matrix.columnVector([0, 0, 0, 1, 1, 1, 2, 2, 2]);

// we will train our model
var rf = new RandomForestClassifier({nEstimators: 1000});
rf.train(X,Y);

// we try to predict the test set
var finalResults = rf.predict(Xtest);
console.log(finalResults)
// Now, you can compare finalResults with the Ytest, which is what you wanted to have.
// output (expected is [0, 0, 0, 1, 1, 1, 2, 2, 2]):
// [ 0, 0, 1, 0, 1, 1, 0, 2, 1 ]
// 5 true predictions (9 values) -> 55% accuracy 

I don't see where is the problem with ml-random-forest, but the results don't seem to be good...

Saved the Random Forest model using toJSON() but while trying to load the saved model it gives an error of
(node:38489) UnhandledPromiseRejectionWarning: TypeError: classifier.load is not a function (classifier being the Random forest classifier)

Hi
It seems that results given by regression example are bad since 2.0.0 (same prediction for each element of array).
Predictions seems to be ok with 1.0.3.

Maybe i missed something ?

Cheers

Hi there.

My use case is to get a movie's genre, and predict the rating that would be given. Since genre are discrete values I considered using Naive Bayes. However since I need to predict the movie rating given, I read that Random Forest can get me the desired result.

I have the following training set which is arrays of inverse document frequencies as follows.
var genreList = ["Biography","Drama","History","Documentary","Action","Comedy","Thriller","Crime","Music","Family","Fantasy","Musical","Animation","Adventure","Sport","Horror","Mystery","Sci-Fi"]
var trainingset = [ [0.1111111111111111,0.05555555555555555,0.2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0.125,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.1111111111111111,0,0,0,0.041666666666666664,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0],[0.1111111111111111,0.05555555555555555,0,0,0,0,0.25,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0.1,1,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0.3333333333333333,0.25,1,0,0,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0.05555555555555555,0,0.125,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0.041666666666666664,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0.05555555555555555,0,0,0,0,0,0,0,0.3333333333333333,0.25,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0.1111111111111111,0.05555555555555555,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0],[0,0,0,0,0.041666666666666664,0.041666666666666664,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0],[0,0.05555555555555555,0,0,0,0.041666666666666664,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0.25,0,0,0.034482758620689655,0,0,0,0],[0,0.05555555555555555,0,0,0,0.041666666666666664,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0,0,0.125,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.1111111111111111,0.05555555555555555,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0,0,0.25,0,0,0,0,0,0,0,0,0.5,0.3333333333333333,0],[0,0.05555555555555555,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.1111111111111111,0.05555555555555555,0.2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0.125,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0,0,0.25,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0.041666666666666664,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0.041666666666666664,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0],[0,0.05555555555555555,0,0,0,0,0,0.1,0,0,0,0,0,0,0,0,0.3333333333333333,0],[0,0,0,0,0.041666666666666664,0.041666666666666664,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0.05555555555555555,0,0,0,0,0,0.1,0,0,0,0,0,0,0,0,0.3333333333333333,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0.05555555555555555,0.2,0,0,0,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0.25,0,0,0.034482758620689655,0,0,0,0],[0.1111111111111111,0.05555555555555555,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0.07692307692307693],[0,0,0,0,0,0.041666666666666664,0,0,0,0.3333333333333333,0,0,0.07692307692307693,0,0,0,0,0],[0,0,0,0.125,0,0,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0.05555555555555555,0,0,0,0,0.25,0,0,0,0,0,0,0,0,0.5,0,0],[0,0,0.2,0.125,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0.1,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0.05555555555555555,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0],[0.1111111111111111,0.05555555555555555,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0.1111111111111111,0,0,0.125,0,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0,0.07692307692307693],[0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0,0.07692307692307693],[0,0,0,0.125,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0.05555555555555555,0.2,0,0,0,0,0,0,0,0,0,0,0.034482758620689655,0,0,0,0],[0,0,0,0,0.041666666666666664,0.041666666666666664,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0],[0,0,0,0,0,0.041666666666666664,0,0,0,0,0,0,0.07692307692307693,0.034482758620689655,0,0,0,0] ]
var predictions = [7,10,8,9,7,3,7,7,10,7,5,6,7,9,8,7,7,7,9,8,7,6,8,8,10,8,7,5,5,8,6,5,6,8,8,2,6,8,7,6,6,5,9,6,6,10,7,7,6,6,10,8,9,7,8,6,8,9,9,7,6,9,7,6,7,7]
However I get the following console error:
Error: input must not be empty
at mean (index.js:12)
at squaredError (utils.js:82)
at Object.regressionError [as regression] (utils.js:106)
at TreeNode.bestSplit (TreeNode.js:57)
at TreeNode.train (TreeNode.js:157)
at DecisionTreeRegression.train (DecisionTreeRegression.js:43)
at RandomForestRegression.train (RandomForestBase.js:95)
at Object. (VJrxxZeJeWDr:131)
at Object.invoke (angular.js:5040)
at $controllerInit (angular.js:11000)

Hi,

thank you for your work on this library.

Other implementations of random forests apply feature bagging at every split during tree generation. So I think it would be better to implement it at the CART level, not here.

Also, "replacement" is a bit of weird option, since it makes no sense to supply a feature more than once to the tree generation. Thus, it basically just randomly shrinks the number of features used for a tree.

• Replace ml-stat with ml-array-mean and ml-array-median (to be created in https://github.com/mljs/array)
• Refactor ml-cart to ES module and use it
• Check and optimize the use of the Matrix package

Hi! Thanks for bringing random forests to the JS world! I was experimenting with making ml-random-forest produce feature importance values and found this strange behavior. It can be reproduced with Iris or any other training data:

var IrisDataset = require('ml-dataset-iris')
var { RandomForestClassifier } = require('ml-random-forest')
var trainingSet = IrisDataset.getNumbers()
var predictions = IrisDataset.getClasses().map((elem) => IrisDataset.getDistinctClasses().indexOf(elem))

var options = {
  maxFeatures: 0.8,
  replacement: true,
  nEstimators: 25
}

var classifier = new RandomForestClassifier(options)
classifier.train(trainingSet, predictions)                                                                                                    
console.log(classifier.indexes)
console.log(JSON.stringify(classifier.estimators[0]) === JSON.stringify(classifier.estimators[1]))

Produces:

[
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ], [ 2, 3, 0 ], [ 2, 3, 0 ],
  [ 2, 3, 0 ]
]
true

So without manually resetting the seed with {seed: undefined} a random forest is not better than a decision tree.

That show two issues:

1. Having default seed is not expected behavior. Random forest results should not be deterministic.
2. When the seed is set, it should not prevent splitting data on different chunks for training. Maybe generate new seeds for each sample with seed+1, seed+2 or something else?

I can create a PR to fix this

Hi , I wonder if it could be possible to handle quantile informations from the library ?

I would be very grateful if you could publish it to npm, thanks!

Please add options.verbosity and logging to track the train process.
Now it can take hours and there is no evidence of how far the model approaches.

Using your example with the Iris dataset I'm only getting around 75% accuracy. I believe a random forest should be able to fit the Iris dataset to around 90%. Do you get the same accuracy as me? I'm concerned because there might be an issue with the implementation of either the forest or the decision tree classifier.

This is caused in the DecisionTreeClassifier load method not finding the 'name' attribute in the json. The solution is to use the toJSON method of the DecisionTreeClassifier when creating the json in RandomForestBase.toJSON.

I solved it locally so if I get the permissions I would create a PR with this change.

I'm trying to get this model to work with a label-to-category dataset but I get this error and the docs didn't help in seeing what was wrong.

/mnt/c/Users/max/Projects/ac-learn/node_modules/ml-matrix/matrix.js:3101                                                                                                          
  throw new RangeError('Inconsistent array dimensions');
  ^
RangeError: Inconsistent array dimensions
at new Matrix (/mnt/c/Users/max/Projects/ac-learn/node_modules/ml-matrix/matrix.js:3101:17)
at Function.checkMatrix (/mnt/c/Users/max/Projects/ac-learn/node_modules/ml-matrix/matrix.js:1373:47)
at RandomForestClassifier.train (/mnt/c/Users/max/Projects/ac-learn/node_modules/ml-random-forest/lib/index.js:154:33)
at Object.<anonymous> (/mnt/c/Users/max/Projects/ac-learn/index.js:17:12)
at Module._compile (internal/modules/cjs/loader.js:799:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:810:10)
at Module.load (internal/modules/cjs/loader.js:666:32)
at tryModuleLoad (internal/modules/cjs/loader.js:606:12)
at Function.Module._load (internal/modules/cjs/loader.js:598:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:862:12)

Here's the code:

const RFClassifier = require('ml-random-foreset').RandomForestClassifier
const LabelDataset = require('./src/dataset')

const trainingSet = LabelDataset.getLabels()
const predictions = LabelDataset.getCategories().map(elem =>
  LabelDataset.getDistinctCategories().indexOf(elem),
)

const options = {
  seed: 3,
  maxFeatures: 0.8,
  replacement: true,
  nEstimators: 25,
}

const classifier = new RFClassifier(options)
classifier.train(trainingSet, predictions)
const result = classifier.predict(trainingSet)
/* eslint-disable no-console */
console.log('result=')
console.dir(result)
/* /* eslint-enable no-console */

I'm fairly new to ML (especially in JS) so I can't tell if something in options is wrong or if I should use ml-cart (which gave the exact same error with the `options from the README) or if the classification model that suits my problem (classifying a set of GitHub labels into categories) is neither Random Forest or Decision Tree.

Please export RandomForestBaseOptions type from this package. It will be useful for type safety when you have an array of options during hyper-parameter optimization.

Hello maintainers,

It would be nice if we have a max_samples option to determine the number of samples to draw from the dataset to train each estimator. The RF classifier implementation of Scikit Learn also has this feature.

I have implemented this in my forked version and can quickly create a PR.

Hi,
I was looking to get some sort of feature importance out of RF, at least the basic Mean Decrease in Impurity.
Quickly browsed through the source code (including ml-cart) but it's clearly above my current skills to implement such a function (as present in scikit-learn).
Any chance it can be implemented in the future?
thanks

Hi guys! Really cool work!!!

A question: can a model be saved and stored/serialized in a file for later usage?

Thanks,
Paul

I'm really enjoying this library. Thank you for all your work!

At the moment, it is possible to train and save a random forest model with toJSON(), but I don't see how to load a saved model. It would be really useful to be able to load pre-trained models.

The error "column indices are out of range error" appears when I try
regression.train([[73, 80],[93, 88]],[152,182])

Uncaught RangeError: column indices are out of range
at checkColumnIndices (util.js:102:11)
at new MatrixColumnSelectionView (columnSelection.js:7:21)
at RandomForestRegression.train (RandomForestBase.js:130:20)
at processRegression (index.html:74:28)
at HTMLButtonElement.onclick (index.html:35:47)

i imported the lib in my backend project but the types.d.ts file is causing compile errors.

`node_modules/ml-random-forest/types/types.d.ts:25:6 - error TS1036: Statements are not allowed in ambient contexts.

25 };
~
node_modules/ml-random-forest/types/types.d.ts:27:22 - error TS2304: Cannot find name 'DecisionTreeClassifier'.

27 type Estimator = DecisionTreeClassifier | DecisionTreeRegression;
~~~~~~~~~~~~~~~~~~~~~~
node_modules/ml-random-forest/types/types.d.ts:27:47 - error TS2304: Cannot find name 'DecisionTreeRegression'.

27 type Estimator = DecisionTreeClassifier | DecisionTreeRegression;
~~~~~~~~~~~~~~~~~~~~~~
node_modules/ml-random-forest/types/types.d.ts:29:36 - error TS2689: Cannot extend an interface 'RandomForestBaseOptions'. Did you mean 'implements'?

29 class RandomForestBase extends RandomForestBaseOptions {
~~~~~~~~~~~~~~~~~~~~~~~
node_modules/ml-random-forest/types/types.d.ts:175:5 - error TS2666: Exports and export assignments are not permitted in module augmentations.

175 export { RandomForestClassifier, RandomForestRegression }
~~~~~~

Found 5 error(s).`

it works fine when i rename the types file to types.ts but it's just temporary.
Note: The library works fine in my front end js project.