The Predictive Accuracy of Full-Rank Variables Vs. Various Types of Factor Scores: Implications for Test Validation

Abstract

A recent condemnation of the use of factor scores as predictors in multiple regression because of the loss in "predictive accuracy" incurred in reducing rank (Kukuk and Baty, 1979) was reexamined from the more important predictive perspective of replication predictive accuracy. Using a computer-based Monte Carlo procedure parallel to that employed in a recent comparison of various types of factor scores (Morris, 1979) the investigator compared the double cross-validation replication predictive accuracies of six types of factor scores with that of full-rank data by utilizing a data set from the literature in which classroom achievement was predicted from affective and cognitive variables. Prediction accuracy was significantly more accurate for each of the six types of factor scores than for fullrank data. Further, corroborative evidence was presented for the superiority of incomplete factor scores over the nonincomplete methods considered. Moreover, implications for reconsidering the sample size typically deemed necessary for factor analysis in predictive situations were offered.EDUCATIONAL AND PSYCHOLOGICAL MEASUREMENT 1980, 40 A recent article in this journal (Kukuk and Baty, 1979) severely criticized the use of factor scores as predictor variables in multiple regression equations for several reasons. Paramount in the rationale for this criticism was a demonstration of the loss in &dquo;predictive accuracy&dquo; incurred in reducing rank. This point was illustrated with an example data set in which demographic characteristics of the family were used to predict students' reading achievement. ' This report was funded in part by

The neuropsychology of borderline personality disorder: A preliminary study on the predictive variance of neuropsychological tests vs. personality trait dimensions

“…Moreover, Dobie et al (1986) and Morris (1980) have shown that if variable correlations are high and the number of variables in one's model is large, the use of factor score regression provides a better solution than the raw score solution because: (1) no serious loss of predictors occur, (2) shrinkage of the factor score R 2 is relatively small, (3) adjustment to the number of df is justifi ed and (4) interpretation of the solution is easier. However, when the correlations between independent variables is weak and the number of independent variables is small, a raw score solution is generally preferred (Dobie et al, 1986;Morris, 1980). Thus, while the sample size is not optimal, we believe our method represents the best approach to examining these relationships.…”

“…When researchers have a large number of strongly correlated independent variables, factor score regression is advocated because it decreases the number of predictors in one's model while simultaneously accounting for multicollinearity problems that can arise from including strongly correlated independent variables (Brown, 2006;Chatterjee & Price, 1977;Dobie, McFarland, & Long, 1986). Moreover, Dobie et al (1986) and Morris (1980) have shown that if variable correlations are high and the number of variables in one's model is large, the use of factor score regression provides a better solution than the raw score solution because: (1) no serious loss of predictors occur, (2) shrinkage of the factor score R 2 is relatively small, (3) adjustment to the number of df is justifi ed and (4) interpretation of the solution is easier. However, when the correlations between independent variables is weak and the number of independent variables is small, a raw score solution is generally preferred (Dobie et al, 1986;Morris, 1980).…”

Methods and Problems of Prediction

“…Various simulation studies were published [Morris and Guertin, 1977;Morris, 1980] which seem to indicate that it can be better to use not the raw values as predictors but factor scores derived from these original data. But against this procedure various plausible arguments have been brought forward [Kukuk and Baty, 1979], In fact, the use of factor scores instead of the original predictors in multiple linear regression at best can be considered as a particular kind of variable selection and depending on the structure of the data other methods of variable selection might be more efficient.…”

Recovering Preipsative Information From Additive Ipsatized Data

“…Cotter and Raju (1982) also found that the estimated factor scores generally do well in predicting population squared cross-validity (the regression weights derived on a sample applied to the entire population). Apart from methodological applications of factor scores, there are also many applied applications using factor scores as independent variables for further data analyses, such as regression, ANOVA, and cluster analysis (e.g., Kemtes & Kemper, 2001;Lastovicka & Thamodaran, 1991;Mitchell & Olson, 1981;Morris, 1980;Williams & Spiro, 1985). Recently, Jöreskog (2000) summarized five possible applications of the estimated factor scores: Despite many applications, the usefulness of factor scores is still controversial because of the issue of indeterminacy.…”