Feature Selection about randomforest-matlab HOT 18 CLOSED

random forests have been used for gene sequencing classification. you can find 
those refs. online.

if you can represent your input data as a matrix and targets as a vector for a 
supervised classification/regression framework, then refer to the tutorial 
files in the package. and also take a look at variable importance 
http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#varimp which 
gives feature importance.

Yes, I can represent input as a matrix. The features are the positions of 
A,C,T,G in the sequence.So I need to specify categorical attributes. The input 
matrix will be in the form of 1's and 0's.It represents the presence or absence 
of features.For eg.
If the input sequence is ACTGCCCGTCATCCC, then the matrix will be in the form
(1,0,1), (1,0,0,1),(1,0,0,0),(0,0,1,1,0) etc..

yup, that would work

this package doesnot do categorical splits for the trees but splits the 
features considering it as a continous value.i think, it wont affect the 
results unless you have many many categories. if R works out better for you and 
you need categorical splits then use this 
http://cran.r-project.org/web/packages/randomForest/index.html from which i 
derived the matlab package.

do tell if you have any questions.

Thank you for the information.this is for my pg project, and i am totally new 
in classification field.And also I am new to R and i just downloaded it. Let me 
go through cran.r-project.If I have queries i will post it.

Is there any restriction in number of fetures?

usually none. i have run datasets with upto 50k features and 1000 examples.

I am trying this code for my classification. I downloaded the precompiled 
windows file. Then i tried to run the compile_windows.m file and got the the 
following error.
I am going to use the precompiled fortran file
If it doesnt work then use cygwin+g77 (or gfortran) to recompile rfsub.f
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 83  empty declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 83  syntax error; found "C" expecting `;' 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 83  unrecognized declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 83  unrecognized declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 100  unrecognized declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 102  empty declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 102  syntax error; found "C" expecting `;' 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 102  unrecognized declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 102  unrecognized declaration 
Error 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 110  unrecognized declaration 
Warning 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 121  empty declaration 
Warning 
C:\USERS\RAKSHMI\DOCUME~1\MATLAB\MAINPR~1\RF_MEX~1.02-\RANDOM~1\RF_CLA~1\SRC\cla
ssRF.cpp: 122  empty declaration 
10 errors, 2 warnings 

  C:\PROGRA~2\MATLAB\R2008B\BIN\MEX.PL: Error: Compile of 'SRC\classRF.cpp' failed. 

??? Error using ==> mex at 213
Unable to complete successfully.

Error in ==> compile_windows at 19
        mex  -DMATLAB -output mexClassRF_train   src/classRF.cpp src/classTree.cpp src/cokus.cpp precompiled_rfsub/win32/rfsub.o
        src/mex_ClassificationRF_train.cpp   src/rfutils.cpp

hi 

if you are using the precompiled version, dont use the compile_windows.m file. 

the compiled mex files are already present in the precompiled package(they 
might be removed if compile_windows.m was ru, so re-extract the files if needed)

just take a look into tutorial_*.m files

[deleted comment]

i got an earlier message of it looking like taking a long time.

you might want to print out the trace while the trees are being created

    extra_options.do_trace = 1; %(Default = 0)

    model = regRF_train(X_trn,Y_trn, 100, 0, extra_options);

Is it possible to do clustering with this code?

hi

the code for clustering is not within the download but you can get the code 
within the svn repository 
http://code.google.com/p/randomforest-matlab/source/checkout

http://code.google.com/p/randomforest-matlab/source/browse/trunk/RF_Class_C/tuto
rial_ClusterRF.m

is a tutorial file. and if you pass in without any labels you can use the 
proximity matrix between examples 
http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#prox

what i am then doing is plotting the first 2 principal components via cmdscale

also take a look at how to interpret proximities for clustering 
http://www.stat.berkeley.edu/~breiman/RandomForests/cc_home.htm#cluster

 its not straightforward clustering but a way of dimensionality reduction. though some have used the proximity matrix in conjunction with other clustering methods (mediod) http://www.iai.uni-bonn.de/~jz/RFclusteringRenalCancer.pdf

I get the following error when running this tutorial file tutorial_ClusterRF.m

??? Error using ==> classRF_train at 192
have to fill stuff here

Hi Flemming

the code for clustering is in the svn and not in the downloads.

you will have to compile the code from the svn to get clustering working

Is it possible to find out which all features were used for classification?

hi

usually all features will be used for classification

if you want to know what are the most important features (and sometimes the top 
some number of features are all what is required for good accuracy), take a 
look into importance (which should be in the tutorial file)

hai abhirana. i want to explore random forest in feature selection process, to 
know how affective this method. have u tried it? do you got the matlab source 
codes for random forest in feature selection? TQ