Prediction‐Oriented Model Selection in Partial Least Squares Path Modeling

Abstract

Partial least squares path modeling (PLS-PM) has become popular in various disciplines to model structural relationships among latent variables measured by manifest variables. To fully benefit from the predictive capabilities of PLS-PM, researchers must understand the efficacy of predictive metrics used. In this research, we compare the performance of standard PLS-PM criteria and model selection criteria derived from Information Theory, in terms of selecting the best predictive model among a cohort of competing models. We use Monte Carlo simulation to study this question under various sample sizes, effect sizes, item loadings, and model setups. Specifically, we explore whether, and when, the in-sample measures such as the model selection criteria can substitute for out-of-sample criteria that require a holdout sample. Such a substitution is advantageous when creating a holdout causes considerable loss of statistical and predictive power due to an overall small sample. We find that when the researcher does not have the luxury of a holdout sample, and the goal is selecting correctly specified models with low prediction error, the in-sample model selection criteria, in particular the Bayesian Information Criterion (BIC) and Geweke-Meese Criterion (GM), are useful substitutes for out-of-sample criteria. When a holdout sample is available, the best performing out-of-sample criteria include the root mean squared error (RMSE) and mean absolute deviation (MAD). Finally, we recommend against using standard the PLS-PM criteria (R 2 , Adjusted R 2 , and Q 2), and specifically the out-of-sample mean absolute percentage error (MAPE) for prediction-oriented model selection purposes. Finally, we illustrate the model selection criteria's practical utility using a well-known corporate reputation model.

Beyond a tandem analysis of SEM and PROCESS: Use of PLS-SEM for mediation analyses!

“…A potential objection regarding the use of PLS-SEM compared with factor-based SEM could be that the method does not offer model modification indices to readily address potential problems of model misspecification as caused by, for example, omitted variables. However, recent research has introduced procedures for comparing PLS path models in terms of model fit and predictive power (Liengaard et al, 2020; Sharma et al, 2019, 2020). While these procedures do not provide a stand-alone assessment of a model, they allow contrasting different model configurations that vary, for example, the position of a moderator in a conditional process model.…”

Structural model robustness checks in PLS-SEM

“…Shmueli et al (2018) use this model and a subset of the data to illustrate Shmueli et al's (2016) PLSpredict procedure. However, Shmueli et al's (2018) presentation draws on a more complex variant of the model by illustrating PLS-SEM's predictive capabilities, which favor the use of a more complex model (Sharma et al, 2019b). 3.…”

“…In addition, the many controversial debates about the method's merits and limitations, witnessed in different research fields (Khan et al, 2019), have increased awareness of it (Petter, 2018). As part of these debates, researchers identified blind spots in PLS-SEM, which methodologists were quick to fill by developing methodological improvements (Franke and Sarstedt, 2019;Henseler et al, 2015;Sharma et al, 2019b).…”

Evaluando la evaluación. Validación mediante PLS-SEM de la escala ATAE para el análisis de las tareas de evaluación

“…In our case we found that small values are achieved in the effect size (Table 9). Finally, to test the predictive power of the model, the PLSpredict procedure (Sharma et al, 2018) was used, obtaining the results presented in Table 10. It is evident that, in all cases, the Q2predict values are above zero and in half of the indicators higher RMSE values are obtained using PLS versus LM, which indicates that the model has an medium predictive power (Shmueli et al, 2019;.…”