Variable importance for machine learning models
Machine learning refers to a broad range of techniques that are designed to model complex relationships among (usually) many variables. These problems are not easily dealt with using more traditional methods such as linear regression. Machine learning techniques are particularly useful for automatically identifying interactions among independent variables.
Traditionally, machine learning algorithms are used to obtain predictions of the outcome variable. However, because they can identify complex relationships between the predictor variables and the outcome, machine learning algorithms can also be useful to identify which of the predictors are most strongly related to the outcome variable. In standard models that are frequently used to make such determinations (e.g., linear regression, logistic regression) variable importance is determined using coefficients. However, machine learning models don’t yield such easy to interpret values meaning that we need to use alternative approaches to assess the importance of predictors with respect to their relationships with the outcome variable. The purpose of this post is to describe several of these methods and to show how you can use them in your own research. As we’ll see, combining these variable importance measures with a machine learning algorithm can provide deeper insights into the relative strength of relationships between the predictors and the outcome. And, as always, I provide you with R code that you can apply directly to your own data. I hope that you enjoy learning the importance of variable importance!!!
