Confirmatory Factor Analyses in Psychological Test Adaptation and Development

Abstract

The importance of providing structural validity evidence for test score(s) derived from psychometric test instruments is highlighted by several institutions; for example, the American Psychological Association (2014) demands that evidence for the validity of an instruments' internal structure and its underlying measurement model must be provided before it is applied in psychological assessment. The knowledge about the latent structure of data obtained with tests addressing the major question "What is/are the construct[s] being measured" by psychological tests under investigation (Ziegler, 2014(Ziegler, , 2020. The study of structural validity is typically addressed with factor analyses when the test scores reflect continuous latent traits. As most submissions to Psychological Test Adaptation and Development (PTAD) deal with the adaptation and further development of existing measures, authors typically test a measurement model that is based on theoretical considerations and prior findings on original versions (or adaptations) of the test under investigation. Our literature review of PTAD's publications showed that more than 90% of the articles contain at least one confirmatory factor analysis (CFA).As editor and reviewers of PTAD, we appreciate that authors are rigorous in providing evidence on the structural validity of their tests' data. However, since PTAD's inception in 2019, we experience that one comment is frequently communicated to authors during the review process, namely, the request to adjust the analytic approach in CFA from maximum likelihood (ML) estimation toward using the mean-and variance-adjusted weighted least squares (WLSMV; Muthén et al., 1997) estimator to account for the ordinal nature of the data that psychological instruments typically generate on the item level. In this editorial, we discuss the rationale behind choosing the WLSMV estimator when analyzing test adaptations and developments that are based on ordinal categorical data and concisely illustrate the problems associated with using the ML estimator (potentially in combination with robust tests of model fit) for such data.

The Brazilian version of the Inventory of Depression and Anxiety Symptoms–Expanded Version: Internal structure, invariance by sex and race, and associations with pathological personality traits.

“…e16 continuous scores. These estimators are respectively appropriate for each type of data (Brauer et al, 2023).…”

Understanding the Clance Impostor Phenomenon Scale Through the Lens of a Bifactor Model

“…We computed the analyses in Mplus 8.8 (Muthén & Muthén, 1998–2019). While Wang and colleagues used the maximum-likelihood estimator, we chose the WLSMV estimator because it accounts for the non-continuous distribution of the responses generated by Likert-type response scales (Brauer et al, 2023). To our knowledge, sample size recommendations are only available for confirmatory bifactor models, and our samples exceed those (≥ 500; Bader et al, 2022).…”

An Analysis of the Hebrew Translation of the OLIW-S to Assess Adult Playfulness

“…Therefore, we fitted a model containing four correlated factors with the item assignment for the four factors, as shown in Proyer et al (2020). We used the WLSMV estimator to account for the ordinal nature of the response data generated by Likert-type rating scales (Brauer, Ranger, et al, 2023). We evaluated the model fit with regard to RMSEA, CFI, TLI, and SRMR.…”