Variable Selection Method for the Identification of Epistatic Models

Abstract

Standard analysis methods for genome wide association studies (GWAS) are not robust to complex disease models, such as interactions between variables with small main effects. These types of effects likely contribute to the heritability of complex human traits. Machine learning methods that are capable of identifying interactions, such as Random Forests (RF), are an alternative analysis approach. One caveat to RF is that there is no standardized method of selecting variables so that false positives are reduced while retaining adequate power. To this end, we have developed a novel variable selection method called relative recurrency variable importance metric (r2VIM). This method incorporates recurrency and variance estimation to assist in optimal threshold selection. For this study, we specifically address how this method performs in data with almost completely epistatic effects (i.e. no marginal effects). Our results show that with appropriate parameter settings, r2VIM can identify interaction effects when the marginal effects are virtually nonexistent. It also outperforms logistic regression, which has essentially no power under this type of model when the number of potential features (genetic variants) is large. (All Supplementary Data can be found here: http://research.nhgri.nih.gov/manuscripts/Bailey-Wilson/ r2VIM_epi/).

Prediction of remission in obsessive compulsive disorder using a novel machine learning strategy

“…There is no gold standard method for determining the threshold that best differentiates signal from noise (Holzinger et al ., 2015). Expert consensus (Strobl et al ., 2009) suggests that it is best not to interpret or compare importance scores but to rely on the relative rankings of the predictors.…”

“…As such, our approach provides an interpretive framework by identifying the set of predictors whose importance scores were consistently (i.e. in 100% of 5000 runs) above a standard threshold (Strobl et al ., 2009; Holzinger et al ., 2015) used for filtering out noise, thereby identifying the predictors that most consistently influence the outcome under study. As noted in Holzinger et al .…”

Advantages of Synthetic Noise and Machine Learning for Analyzing Radioecological Data Sets

“…Consequently, it was shown that repeating the machine learning analysis several times with different random number seeds is more reliable than a single run [10, 11]. Specifically, running a machine learning algorithm multiple times with different seeds generates a distribution of VIMr values across runs.…”

“…For tree-based machine learning methods such as RF, overfitting generally occurs if the trees are allowed to continue splitting to purity [10, 11]. In other words, if the trees are allowed to become very complex, they are likely to “overreact” to noise in the data.…”

“…In other words, if the trees are allowed to become very complex, they are likely to “overreact” to noise in the data. A simple solution for this problem involves preventing the terminal nodes in any tree from containing less than about 10% of the sample size [10, 11]. This critical value (Nc) can be calculated simply by rounding the product 0.1×N, where N is the sample size.…”