Supervised machine learning methods in psychology: A practical introduction with annotated R code

Abstract

Machine learning methods for prediction and pattern detection are increasingly prevalent in psychological research. We provide an introductory overview of machine learning, its applications, and describe how to implement models for research. We review fundamental concepts of machine learning, such as prediction accuracy and out-ofsample evaluation, and summarize standard prediction algorithms including linear regressions, ridge regressions, decision trees, and random forests (plus additional algorithms in the supplementary materials). We demonstrate each method with examples and annotated R code, and discuss best practices for determining sample sizes; comparing model performances; tuning prediction models; preregistering prediction models; and reporting results. Finally, we discuss the value of machine learning methods in maintaining psychology's status as a predictive science. 1 | INTRODUCTION Psychologists are increasingly interested in adopting powerful computational techniques from the field of machine learning to accurately predict real-world phenomena (see Yarkoni & Westfall, 2017). The current work introduces machine learning as a collection of methods and tools that can be used in prediction. We review fundamental concepts of machine learning, discuss its relationship with standard psychological methods, and give This is an open access article under the terms of the Creative Commons Attribution-NonCommercial License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited and is not used for commercial purposes.

Which Cognitive Factors Predict L2 Grammar Learning: Cognitive Control, Statistical Learning, Working Memory, or Attention?

“…This is because traditional regression analysis only inferences the result patterns from a given dataset while machine learning can not only extract the patterns from a given dataset but also predict the patterns in a new dataset. In other words, the traditional approach addresses the problem concerning whether the independent variables predict the dependent variables, while machine learning can address the question of how well the independent variables predict the dependent variables ( Rosenbusch et al, 2021 ). Therefore, this renders more reliability and generalizability of the results with machine learning analysis, making the results more replicable and generalizable ( Orrù et al, 2020 ).…”

“…We first constructed the base models with the default hyperparameters of the algorithms, then we tuned the hyperparameters with the grid search method ( LaValle et al, 2004 ). We subsequently constructed the best models with the best parameters obtained via the grid search method and compared the r 2 values of these models to identify the best model ( Rosenbusch et al, 2021 ). However, owing to the small sample size of the study, even with the best models constructed, we still obtained a negative r 2 value for day 2 to day 4 analyses.…”

Ecological dimensions explain the past but do not predict future changes in trust.

“…For example, in contrast to predictive models, explanatory models typically include theoretically relevant confounding variables even when these variables turn out to be unrelated to the outcome (Shmueli, 2010). At the statistical level, explanatory models might be overinformed by idiosyncratic features of the dataset used to test them (i.e., overfitting), which could render them less well suited to predict outcomes in new data (Rosenbusch et al, 2020). Finally, explanatory models often prioritize elegance and simplicity, while human behavior is multidetermined and potentially idiosyncratic.…”

Changing channels? A comparison of Fox and MSNBC in 2012, 2016, and 2020

“…For Compartmentalization these were organized, tidy, well-defined, and clear versus disorganized, untidy, and vague. 5 For those using R, see Rosenbusch et al (2021) for a detailed introduction to the use of random forests and related techniques.…”