A Monte Carlo evaluation of growth mixture modeling

Abstract

Growth mixture modeling (GMM) and its variants, which group individuals based on similar longitudinal growth trajectories, are quite popular in developmental and clinical science. However, research addressing the validity of GMM-identified latent subgroupings is limited. This Monte Carlo simulation tests the efficiency of GMM in identifying known subgroups (k = 1–4) across various combinations of distributional characteristics, including skew, kurtosis, sample size, intercept effect size, patterns of growth (none, linear, quadratic, exponential), and proportions of observations within each group. In total, 1,955 combinations of distributional parameters were examined, each with 1,000 replications (1,955,000 simulations). Using standard fit indices, GMM often identified the wrong number of groups. When one group was simulated with varying skew and kurtosis, GMM often identified multiple groups. When two groups were simulated, GMM performed well only when one group had steep growth (whether linear, quadratic, or exponential). When three to four groups were simulated, GMM was effective primarily when intercept effect sizes and sample sizes were large, an uncommon state of affairs in real-world applications. When conditions were less ideal, GMM often underestimated the correct number of groups when the true number was between two and four. Results suggest caution in interpreting GMM results, which sometimes get reified in the literature.

Psychopathology and resilience following strict COVID-19 lockdowns in Hubei, China: Examining person- and context-level predictors for longitudinal trajectories.

“…This may negatively impact our data quality. Third, the use of LGMM can sometimes identify the wrong number of groups according to the levels of skew, kurtosis, intercept effect size, and group proportions (Depaoli et al, 2019; Shader & Beauchaine, 2021). Following recommendations by Nylund-Gibson and Choi (2018), we excluded solutions with infrequent classes (<5%) even when some fit indices may support solutions with more classes, because small classes tend to be unstable.…”

“…However, we selected the two-class solution as optimal because subsequent class solutions had at least one class with low prevalence (< 5%). The two classes had reasonably large intercept effect sizes (Cohen's d = 1.55) easing concern of non-normal data driving grouping solutions according to simulation studies (Depaoli et al, 2019;Shader & Beauchaine, 2021). All of the selected models produced the best AIC and SSABIC among models without any class of low prevalence (<5%).…”

“…In the final models, both slope and intercept variances were allowed to be freely estimated and quadratic parameters were nonsignificant and thus removed to facilitate model convergence. Because recent simulation studies (Depaoli et al, 2019;Shader & Beauchaine, 2021) indicated that LGMM on nonnormal data can over-identify latent groups, we also examined the levels of skewness and kurtosis of our data (Supplemental Material 2). Model solutions from 1 to 5 classes were compared by model fit indices, including Akaike information criteria (AIC), sample-size adjusted Bayesian information criteria (SSABIC), Entropy, and Bootstrap Likelihood Ratio Test (BLRT).…”

Understanding posttraumatic stress trajectories in adolescent females: A strength-based machine learning approach examining risk and protective factors including online behaviors

“…Females have been shown to be disproportionately impacted by exposure to online sexual contents (Kohut & Štulhofer, 2018) and have been more likely to be classified as high-risk internet users as compared to males (Victorin et al, 2020). Lastly, recent simulation studies (Depaoli et al, 2019;Shader & Beauchaine, 2021) indicated that group identification in LGMM can be impacted by non-normally distributed data. However, this concern is attenuated when intercept effect sizes are large, which is the case in the present study (e.g., intercept effect size of resilient and chronic: Cohen's d = 1.02).…”

“…However, this concern is attenuated when intercept effect sizes are large, which is the case in the present study (e.g., intercept effect size of resilient and chronic: Cohen's d = 1.02). Given that LGMM has been shown to be more effective when intercept effect sizes were large (Shader & Beauchaine, 2021), examining the two latent groups with the least (i.e., resilient) and most (i.e., chronic) prevalent PTSS avoids differentiating latent groups that were potentially artifacts of the methodology. However, since the non-normal distribution of PTSS in the present study (range of skewness Time 1-Time 3 = 0.48-0.69, range of kurtosis Time 1-Time 3 = 2.11-2.32) remains a concern for class misidentification (Depaoli et al 2019), it is critical to consider this limitation when interpreting findings.…”