xgess / new-random-forest-idea Goto Github PK

Alex's Crazy Random Forest Idea
===============================

Random Forests typically promote diversity between trees by using a number of predictors to 
consider at each node (M) which is less than the total number of predictors (P). M seems to 
range from about 0.3 P to about 0.6 P, but is constant in the entire forest. This 
implementation chooses M from a distribution. 


In an attempt to increase diversity without losing accuracy 
(for each of the trees) this code draws M from a distribution before choosing the predictor 
variables to use at each node. My thought is that this will allow some nodes in each tree to 
be much more accurate than other nodes. Since the trees are grown out completely (i.e. not
pruned) this might make a difference. Tests indicate nothing conclusive. This implementation 
was a little better for some data sets and a little worse for others. All the data and tests 
are in this repo. The professor of the class and I both agree it is worth exploring further 
with more data and a faster implementation perhaps in something like C that can then be 
hooked into R. If I wind up doing that, I'll add a link here. 

The main code (building and traversing of the trees) is in randomForestM.R and the test scripts 
are in the files that begin with overnight. There are comparisons with the standard random forest 
code in those test scripts.