Selecting the “Best” Factor Structure and Moving Measurement Validation Forward: An Illustration

Abstract

Despite the broad literature base on factor analysis best practices, research seeking to evaluate a measure's psychometric properties frequently fails to consider or follow these recommendations. This leads to incorrect factor structures, numerous and often overly complex competing factor models and, perhaps most harmful, biased model results. Our goal is to demonstrate a practical and actionable process for factor analysis through (a) an overview of six statistical and psychometric issues and approaches to be aware of, investigate, and report when engaging in factor structure validation, along with a flowchart for recommended procedures to understand latent factor structures; (b) demonstrating these issues to provide a summary of the updated Posttraumatic Stress Disorder Checklist (PCL-5) factor models and a rationale for validation; and (c) conducting a comprehensive statistical and psychometric validation of the PCL-5 factor structure to demonstrate all the issues we described earlier. Considering previous research, the PCL-5 was evaluated using a sample of 1,403 U.S. Air Force remotely piloted aircraft operators with high levels of battlefield exposure. Previously proposed PCL-5 factor structures were not supported by the data, but instead a bifactor model is arguably more statistically appropriate.

Exploring the structure of narcissism: Toward an integrated solution

“…The flexibility of the potential factor solutions is maximized by using narcissism items rather than scales, subscales, or parcels. Furthermore, by utilizing an inclusive pool of over 200 items, the likely number of indicators for each factor is increased along with factor stability (Marsh et al, ; Schmitt, Sass, Chappelle, & Thompson, ). The first factor is expected to be indicative of what is most common to the narcissism scales, a grandiose sense of self‐importance.…”

Evaluating the higher-order structure of the Profile of Emotional Competence (PEC): Confirmatory factor analysis and Bayesian structural equation modeling

“…Because EC facets are positively related to each other [28], they will be hierarchically structured. Clarification of the higher-order structure of individual differences is important because it can provide a parsimonious summary of the vast complexity of human nature [38]. Given that the above 10 core EC facets are categorized into a 2 (type of target) × 5 (type of competence) framework, we can assume six possible structures.…”

“…To evaluate the best factor structure, we followed the flowchart recently proposed by Schmitt, Sass [38]. They encourage researchers to start by conducting dimensionality analyses (e.g., parallel analysis, minimum average partial test, exploratory graph analysis); then, if theoretical candidate factor structures exist, they recommend confirmatory factor analysis (CFA).…”

“…This study aimed to evaluate the higher-order structure of the PEC using Schmitt, Sass [38]’s guidelines. As recommended in their flowchart, we started with dimensionality analyses, followed by traditional CFA and BSEM.…”

Utopian Impulse: An Individual-differences Approach to Transformative Social Change

“…The Kaiser-Meyer-Olkin measure of sampling adequacy, which was .99, and Bartlett's test of sphericity, which was significant, χ 2 (1770) = 37839.86, p < .001, provided a strong evidence for the suitability of the data for EFA (Beavers et al, 2013). To determine how many factors to retain, we first performed a parallel analysis (Hayton, Allen, & Scarpello, 2004;Horn, 1965;Schmitt, Sass, Chappelle, & Thompson, 2018), which showed that five factors should be retained. We therefore performed a maximum likelihood (ML) EFA (Table 1) with five factors and used an oblique rotation-promax-given that we expected the factors to be correlated because they belong to the same theoretical construct and because this rotation typically produces a clean factor structure (Hendrickson & White, 1964;Schmitt & Sass, 2011).…”