Published February, 2018
-
Basic Selection Methods (Univariate) - Rely on characteristics of data
- Removing Constant Features
- Removing Quasi-Constant Features
- Removing Duplicated Features
-
Correlation Feature Selection (Multivariate) - Rely on characteristics of data
- Removing Correlated Features
- Basic Selection Methods + Correlation - Pipeline
-
Filter Methods: Univariate Statistical Methods - Rely on characteristics of data
- Mutual Information
- Chi-square distribution
- Anova
- Basic Selection Methods + Statistical Methods - Pipeline
-
Filter Methods: Other Methods and Metrics
- Univariate roc-auc, mse, etc
- Method used in a KDD competition - 2009
-
Wrapper Methods - Uses predictive machine learning model and uses greedy algorithm
- Step Forward Feature Selection
- Step Backward Feature Selection
- Exhaustive Feature Selection
-
Embedded Methods: Linear Model Coefficients - Featuer Selection as part of the model construction process
- Logistic Regression Coefficients
- Linear Regression Coefficients
- Effect of Regularization on Coefficients
- Basic Selection Methods + Correlation + Embedded - Pipeline
-
Embedded Methods: Lasso - Featuer Selection as part of the model construction process
- Lasso
- Basic Selection Methods + Correlation + Lasso - Pipeline
-
Embedded Methods: Tree Importance - Featuer Selection as part of the model construction process
- Random Forest derived Feature Importance
- Tree importance + Recursive Feature Elimination
- Basic Selection Methods + Correlation + Tree importance - Pipeline
-
Hybrid Feature Selection Methods
- Feature Shuffling
- Recursive Feature Elimination
- Recursive Feature Addition