Factor analyzing ordinal items requires substantive knowledge of response marginals.

Abstract

In the social sciences, measurement scales often consist of ordinal items and are commonly analyzed using factor analysis. Either data are treated as continuous, or a discretization framework is imposed in order to take the ordinal scale properly into account. Correlational analysis is central in both approaches, and we review recent theory on correlations obtained from ordinal data. To ensure appropriate estimation, the item distributions prior to discretization should be (approximately) known, or the thresholds should be known to be equally spaced. We refer to such knowledge as substantive because it may not be extracted from the data, but must be rooted in expert knowledge about the data-generating process. An illustrative case is presented where absence of substantive knowledge of the item distributions inevitably leads the analyst to conclude that a truly two-dimensional case is perfectly one-dimensional. Additional studies probe the extent to which violation of the standard assumption of underlying normality leads to bias in correlations and factor models. As a remedy, we propose an adjusted polychoric estimator for ordinal factor analysis that takes substantive knowledge into account. Also, we demonstrate how to use the adjusted estimator in sensitivity analysis when the continuous item distributions are known only approximately.

Overfactoring in rating scale data: A comparison between factor analysis and item response theory

“…This currently is the common setup in applied psychometrics, and this article is oriented to practitioners that analyze ordinal data with the common methods. However, there is also an active line of research that questions the normality of factors and errors ( Grønneberg and Foldnes, 2022 ; Jobst et al, 2022 ; Manapat and Edwards, 2022 ). The validity of the conclusion from our simulations is contingent upon the normality and the other assumptions involved in the categorical factor model.…”

“…In the simulation, we considered only the case in which latent normality holds. This assumption is not tenable in the empirical application, and the psychometric literature shows that this might be a source of imprecision in the estimates of polychoric correlations ( Grønneberg and Foldnes, 2022 ). All in all, we would recommend the solution with a small number of factors for parsimony and the questionability of polychoric-based methods in this sample.…”

“…He found that mixed skewness is an important driver for the problem of overfactoring in the context of exploratory FA. Other relevant studies are those by Foldnes and Grønneberg (2022) and Grønneberg and Foldnes (2022), who found that the dissimilarity between the distribution of the latent responses and the assumed normal distribution leads to a distortion in the polychoric correlations and, ultimately, in the results of the categorical factor model. Olsson (1979) also found that overfactoring is associated with the magnitude of the factor loadings, occurring that higher factor loadings are associated with an increased chi-square value and, consequentially, lead to a worse fit.…”

“…He found that mixed skewness is an important driver for the problem of overfactoring in the context of exploratory FA. Other relevant studies are those by Foldnes and Grønneberg (2022) and Grønneberg and Foldnes (2022) , who found that the dissimilarity between the distribution of the latent responses and the assumed normal distribution leads to a distortion in the polychoric correlations and, ultimately, in the results of the categorical factor model.…”

The Value of Sustainable Tourism Destinations in the Eyes of Visitors

“…Although most of the literature about perceived value estimates Structural Equation Models (SEM) for continuous data, we recur to an Ordinal SEM (OSEM, see [52]) because the distributional characteristics of our data, examined through a battery of tests (Shapiro-Wilk's test for normality [53]; D'Agostino's test of skewness [54]; Geary's measure of kurtosis [55]) recommend not to rely on the linear approximation [56,57]. In general, an OSEM can be described by the following set of equations:…”