On the Fallibility of Principal Components in Research

Abstract

The measurement error in principal components extracted from a set of fallible measures is discussed and evaluated. It is shown that as long as one or more measures in a given set of observed variables contains error of measurement, so also does any principal component obtained from the set. The error variance in any principal component is shown to be (a) bounded from below by the smallest error variance in a variable from the analyzed set and (b) bounded from above by the largest error variance in a variable from that set. In the case of a unidimensional set of analyzed measures, it is pointed out that the reliability and criterion validity of any principal component are bounded from above by these respective coefficients of the optimal linear combination with maximal reliability and criterion validity (for a criterion unrelated to the error terms in the individual measures). The discussed psychometric features of principal components are illustrated on a numerical data set.

On the Unlikely Case of an Error-Free Principal Component From a Set of Fallible Measures

“…We assume that a set of observed measures is given, whereby at least two of them contain measurement error (with positive variance), as it will often be the case in the majority of contemporary studies in the educational and behavioral disciplines (see Raykov et al, 2016). We denote these measures by X 1 , .…”

“…(Analyzing instead the correlation matrix does not alter the following developments, findings, and interpretations.) We stress that the remaining discussion evolves exclusively at the population level, like that in Raykov et al (2016).…”

“…, k ). Using the classical test theory decomposition for the manifest measures (note that if the s th of them, X s , is error-free, then e s = 0 is valid for all cases; 1≤ s ≤ k ), evidently the corresponding decomposition of Y j is as follows (e.g., Raykov et al, 2016):…”

“…That is, in practice it is (in effect) impossible that Equation (5) will hold in a given empirical study (even if conducted at the population level). Therefore, it is practically impossible for a PC, which is extracted from a set of observed measures where at least two are fallible, to have perfect reliability regardless whether the error terms of these fallible measures correlate (for the case of uncorrelated errors, see Raykov et al, 2016; see also Note 1).…”

“…Recently, Raykov, Marcoulides, and Li (2017) pointed out that if (a) at least one measure is fallible in a set being analyzed via PCA and (b) the pertinent measurement errors are uncorrelated (when two or more observed variables are not error-free), then no PC can have lower error variance than that in the fallible measure with minimal error variance. An important assumption of their developments was that of no correlated error sources (measurement errors) in the analyzed variables, which seems to be advanced explicitly or implicitly similarly in many empirical studies.…”