Treating Factor Interpretations as Hypotheses

Abstract

Factor analysis is used extensively in the construction of psychometric scales. It may appear to be a mathematically precise and objective technique, but it involves many subjective choices and its results require subjective interpretation. Factor interpretations are hypotheses that should be tested, yet often are accepted simply on the basis of post hoc expert judgment. Untested judgments about the meaning of factors sometimes result in mislabeling of published scales and misinterpretation of research findings. This article cites several such cases to illustrate the need for testing factor-interpretation hypotheses and the difficulty of doing so without bias. Techniques for generating and testing such hypotheses are then reviewed.

Defining distinct negative beliefs about uncertainty: Validating the factor structure of the Intolerance of Uncertainty Scale.

“…A survey of the items that loaded on Factor 1 suggested that this factor encompasses the beliefs that being uncertain impairs behavior and reflects badly on an individual’s character; accordingly, this factor was labeled “Uncertainty Has Negative Behavioral and Self-Referent Implications.” The second set of items reflected the belief that the future should be predictable and that unpredictability is unfair and therefore distressing; accordingly, Factor 2 was labeled “Uncertainty Is Unfair and Spoils Everything.” Both factor labels therefore showed face validity for the items they comprise. Nonetheless, Tracy (1990) has recommended that additional methods be employed to assess the validity of factor labels.…”

“…Both factor labels therefore showed face validity for the items they comprise. Nonetheless, Tracy (1990) has recommended that additional methods be employed to assess the validity of factor labels.…”

“…Finally, an examination of the developmental trajectories of these beliefs may have important implications for prevention and treatment. Tracy (1990) noted that factor labels should be regarded as hypotheses to be subjected to further testing and scrutiny. Although the current study provided preliminary evidence of the construct validity of these factors, first by confirming the twofactor structure of the IUS in an independent sample and second by assessing the factors' correlations with symptom measures, the construct validity of the proposed subscales remains to be firmly established using additional experimental methods.…”

Information Theory, Machine Learning, and Bayesian Networks in the Analysis of Dichotomous and Likert Responses for Questionnaire Psychometric Validation

“…Secondly, it avoids the subjec-tivity in factor loadings interpretation (Fairweather, 2001;P. Jordan & Spiess, 2019;Tracy, 1990;Wrigley, 1958). Our approach overcomes this aspect by focusing on items instead of latent factors.…”