Improving Practices for Selecting a Subset of Important Predictors in Psychology: An Application to Predicting Pain

Abstract

In this paper we address the problem of selecting important predictors from some larger set of candidate predictors. Standard techniques are limited by lack of power and high false positive rates. A Bayesian variable selection approach used widely in biostatistics, stochastic search variable selection, can be used instead to combat these issues by accounting for uncertainty in the other predictors of the model. In this paper we present Bayesian variable selection to aid researchers facing this common scenario, along with an online application (https://ssvsforpsych.shinyapps.io/ssvsforpsych/) to perform the analysis and visualize the results. Using an application to predict pain ratings, we demonstrate how this approach quickly identifies reliable predictors, even when the set of possible predictors is larger than the sample size. This technique is widely applicable to research questions that may be relatively data-rich, but with limited information or theory to guide variable selection.

Searching for Prosociality in Qualitative Data: Comparing Manual, Closed–Vocabulary, and Open–Vocabulary Methods

“…We used the R code available from Bainter, McCauley, Wager, and Losin (n.d., in press) to run the SSVS models. We based results on 20 000 regression models (the first 5000 were discarded warm‐ups) that sample different combinations of the 102 predictors.…”

Modelling Predictors of Gambling Harms with Stochastic Search Variable Selection (SSVS)

“…Identifying the relative importance of the dimensions of those factors as predictors of experiencing harm may lead to targeted interventions designed to attenuate the negative consequences. The purpose of the present study was to introduce stochastic search variable selection (SSVS) as a procedure to achieve that goal 10 .…”

“…There were 10 statistically significant bivariate correlations at the 1% level of significance (p < .01). We planned to include as many predictors for the regression model as would have been selected based on bivariate correlations 10 . Therefore, for the regression model selected based on MIPs, the dimensions with the top 10 MIPs were to be included.…”

“…However, bivariate tests may select a set of predictors that are redundant due to the predictors being highly intercorrelated. Furthermore, subset models based on bivariate correlations may include predictors that are meaningfully related to the criterion only when the model controls for one or more other predictors 10 .…”

“…One statistical procedure that identifies important predictors is stochastic search variable selection (SSVS). SSVS has not been commonly implemented in psychological research but offers a method that is not subject to the limitations of methods that rely on bivariate relationships or stepwise regression 10 . SSVS samples thousands of high probability regression models made up of subsets of the predictors and returns the proportion of models that include each predictor, otherwise known as the marginal inclusion probabilities (MIPs) of the predictors.…”