The present study compared two approaches to scoring a Psychological Climate Questionnaire— an empirical keying of items using item analysis and a rational approach which focused on identifying the underlying constructs measured by the questionnaire. The approaches were compared with respect to prediction of performance criteria and the ability to ascertain the theoretical underpinnings of the instrument. Questionnaires were completed by 398 male firemen in a large metropolitan area. Results demonstrated that a summation of item scores to reflect carefully designed a priori constructs did not necessarily result in weaker prediction of performance criteria. Similarities of the present study with previous work are discussed. Three popular approaches to scoring a set of questionnaire items (e.g., personality inventories, biographical inventories, attitude surveys) for which there is no single correct response have been evident in the psychological literature (cf. Sparks, 1969;Nunnally, 1967). These three approaches have been labeled the external (or empirical) approaclr, the internal approach, and the intuitive approach (see Goldberg, 1972, for an extensive review of these approaches).The empirical approach places initial emphasis on the prediction of an external criterion, typically using an empirical keying procedure based on item-criterion relationships. Understanding of underlying relationships or constructs accounting for the prediction has received secondary or no emphasis. This approach has been thought to maximize prediction of an external criterion (Guion, 1965). It has been criticized, however, because a capitalization on unique item variance often results in criterionspecific scoring keys with low internal consistency if the criterion is heterogeneous (Goldberg, 1972;Guion, 1965;Nunnally, 1967). The resulting lack of internal consistency often hinders determination of the theoretical composition or construct validity of the scoring keys (Baehr & Williams, 1967; Matteson et al., 1969;Nunnally, 1967). Thus, little understanding of conceptual relationships between the items and the criterion may be conveyed by this approach.The internal approaclr uses the internal structure of the questionnaire items as the sole source of information to determine where and how items are to be used in the construction of item composites. Typically, this has been accomplished through the use of factor-analytic procedures or a strategy directed toward maximization of internal consistency (e.g., Loevinger, Gleser, & Dubois, 1953).